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Synapses, membranes and neurotransmitter play an important role in processing information 
in the nervous system. We do not know, however, what biophysical mechanisms are 
critical for neuronal computations, what elementary information processing operations they 
implement, and which sensory or motor computations they underlie. In this paper, we 
outline an approach to these problems. We review a number of different biophysical 
mechanisms, such as synaptic interactions between excitation and inhibition, dendritic 
spines, non-impulse generating membrane nonlinearities and transmitter-regulated voltage 
channels. For each one, we discuss the information processing operations that may be 
implemented, Ali of these mechanisms act either within a few milliseconds, such as the 
action potential or synaptic transmission, or over several hundred milliseconds or even 
seconds, modulating some property of the circuit. In some cases we will suggest specific 
examples where a biophysical mechanism underlies a given computation. In particlar, we will 
discuss the neuronal operations, and their implementation, underlying direction selectivity 
in the vertebrate retina. 
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Abstract 


Synapses, membranes and transmitters play an important role in processing information in 
the nervous system. We do not know, however, what biophysical mechanisms are critical for 
neuronal computations, what elementary information processing operations they implement, 
and which sensory or motor computations they underlie. In this paper, we outline an 
approach to these problems. More precisely 

(1) we review the mechanisms underlying impulse initiation and conduction, repetitive firing 
of impulses and the failure of impulses to propagate past axonal branch points, analyzing 
some of the underlying operations. 

(2) We discuss chemical and electrical transduction at the synapse. The operation performed 
by a chemical synapse is non-linear amplification and is equivalent to a non-reciprocal 
electrical element. Electrical synapses are equivalent to linear and non-linear resistances, 
such as diodes. 

(3) We introduce postsynaptic interaction between conductance changes. In the specific 
case of excitation and shunting inhibition, the operation implemented is the analog equivalent 
of an AND-NOT gate or a veto operation. This mechanism may be the key component in 
the circuitries computing direction of motion in the retina and depth in cortical cells. 

(4) We analyze the function of dendritic spines. Two of the operations that may be 
implemented by dendritic spines are changing the synaptic weight of a synapse and 
subserving a very specific interaction between excitation and inhibition. Examples of 
possible computations are information storage in the cortex and the selective suppression 
of visual input in the LGN. 

(5) We consider the role of quasi-active membranes, i.e., time- and voltage-dependent 
channels. Short of underlying spiking behavior, they can perform different filtering operations, 
acting as resonant filters. An example of a specific computation is the tuning of hair cells 
to acoustic inputs. 

(6) Finally we consider transmitter regulation of voltage dependent channels and the action 
of neurotransmitters at large distances. We discuss how the non-classical action of these 
neuromodulatory substances may affect the circuitry in a very specific manner over long 
time-courses. 

We will conclude by discussing a possible distinction between biophysical mechanisms 
acting over short and long time ranges. Finally, we will present a new biophysical modei 
of analog computation which is very different from the model of computation provided by 
present digital computers and appears to be very suggestive in terms of the nervous system. 
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0. Introduction 

This paper is based on the belief that brains are very sophisticated computing machines, 
and that one of the ultimate goals of brain science is to understand the computations 
and information processing they perform. A study of the role of synapses, membranes 
and cells in information processing tasks is therefore part of this broad enterprise. But 
this is not the only motivation for such a study. In recent years, computational studies 
have provided promising — although far from complete — theories of the computations 
necessary for sensory processing (for partial reviews see Marr, 1982 and Poggio, 1984). 
Despite their initial success, we have now come to realize that computational theories, even 
complemented by psychophysical experiments, have inherent limitations in understanding 
the brain. A given computation, for instance the computation of stereo depth or motion, 
can in general be performed by several different algorithms. These algorithms depend not 
only on the nature of the computation itself, but also on the properties and limitations of 
the hardware in which the algorithm is implemented. 

As a consequence, for bridging the gap between the computational theories and the 
biological data, we must first understand how the elementary computations are performed 
in neural hardware. Neurobiologists have shown in recent years with such experimental 
findings as dendrodendritic synapses, gap junctions, dendritic spikes, nonsynaptic release 
of neural transmitters and a variety of voltage- and transmitter-dependent channels that the 
complexity of the processing that takes place within a single neuron may be far greater 
than previously presumed (see Schmitt and Worden (1979) for a thorough review of this 
developement). An understanding of the role of these biophysical mechanisms in biological 
information processing is necessary, in order to understand the implementation of specific 
computations in neuronal hardware. 

In Computer Science, work on the physics of computation attempts to characterize the 
physical mechanisms that can be exploited to perform elementary information processing 
operations in technical systems (Mead and Conway, 1980). These mechanisms constrain 
in turn the types of operation which can be exploited for computing. A case in point is 
the so-called metal - oxide - semiconductor (MOS) technology used in integrated circuits. 
Due to the physics of the MOS transistor, the basic logic circuit is an inverter, whose 
output is the complement of its input. NAND and NOR logic circuits can be constructed 
as simple expansions of the basic inverter circuit. Therefore, very large scale integrated 
(VLSI) systems in MOS technology use NAND and NOR logic (Mead and Conway, 1980). 
We believe that a biophysics of computation is now needed for understanding the role of 
neurons, synapses and membranes in information processing in biological systems. 

Simply stated, the questions that we are asking are: what is the hardware of the brain? Where 


2 




Koch and Poggio 


Biophysics of Computation 


are the elementary information processing operations performed? What is the equivalent in 
t nervous systems of transistors and gates? In this paper, we outline our initial approach to 

these questions. We review briefly some of the biophysical mechanisms, both in synapses 
and membranes, that may have a role in information processing. For each one, we discuss 
the information processing operations that may be implemented. In some cases, we will 
also suggest specific computations, such as the computation of the direction of motion, 
where the given operation may have an important role. Table 1 gives a list of biophysical 
mechanisms, of corresponding information processing operations and of specific examples 
of computations where each mechanism may be used. The list is not intended to be 
exhaustive. 

This paper is speculative. Its main goal is to state the problem — what are the biophysical 
mechanisms underlying information processing and how are these mechanisms used to 
perform specific computations — and to stimulate experimental and theoretical research 
that can lead to its answer. As we will discuss later, many biophysical mechanisms may have 
only an indirect role in computations in the nervous system and be, so to speak, side effects 
of cell’s properties. The situation is not much different in our present computers: many of 
the components are dedicated to functions, such as supplying power, that are only indirectly 
relevant to the elementary information processing operations. Ultimately, only experimental 
work can answer which biophysical mechanisms are used in information processing, and 
how. Computational considerations in conjunction with computer simulations can help, 
however, to direct effectively experimental work. 

In this paper we consider several biophysical mechanisms of neuronal membranes and 
synapses. For each one we discuss the information processing operation that it may 
implement. For instance, the mechanism leading to action potentials performs a threshold 
operation similar to the analog-to-digital conversion performed by a Schmitt-trigger. A 
subset of these operations is sufficient to synthesize a universal computer. All together they 
provide the nervous system with a powerful set of basic hardware operations in terms of 
which simple and complex computations and algorithms can be implemented. Whenever 
possible, we speculate about a specific instance of computation where the biophysical 
mechanism may play a central role. The best example is probably the computation of the 
direction of motion in retinal ganglion cells. We suggest that the underlying mechanism is 
the interaction of excitatory and inhibitory conductance changes of postsynaptic origin. 

Section 1 considers the biophysical mechanisms of spike initiation and conduction and the 
corresponding information processing operations. It also describes a specific example of a 
computation (for motor control) tfllt exploits the mechanism of differential spike conduction 
in branching axonal trees. Synapses are discussed in section 2. We first characterize from 
the point of view of information processing chemical synapses, then electrical synapses 
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and finally the postsynaptic interactions between synaptic conductance changes. Several 
examples of computations are suggested. Experiments for testing these hypotheses are also 
described. Dendritic spines and their possible role in information storage and processing 
are the subject of section 3. The mechanisms of non-spiking voltage- and time-dependent 
channels are introduced in section 4.1. As an example of a computation, we describe the 
resonant tuning of hair cells in the vertebrate cochlea. Transmitter regulation of voltage 
dependent channels is the subject of section 4.2 and neuropeptides of section 4.3. The final 
section discusses the distinction between biophysical mechanisms acting in the millisecond 
range, such as conductance changes or impulse initiation, and the class of mechanisms 
which modulate the processing of information via these fast mechanisms on a long time 
scale. We will also briefly allude to a novel model of information processing in analog 
electrical or chemical networks. Such a model could be easily implemented in neuronal 
hardware, suggesting a style of neuronal computation very different from threshold models 
of the McCulloch and Pitts type. 



Lis^ -of SOME NEURONAL OPERATIONS AND THE UNDERLYING BIOPHYSICAL MECHANISMS 



1 







£ 





*H 





£ 





O 





*H 

r£ 




4-J 

CO 




£ 

•H 

s 



a 

4-1 

o 



*H 

PH 

•H 



£ 

£ 

3 



£ 

U 

3 



g 

a 

3 



g 


3 



O 

£ 

a 



g 

*H 

a 





o 



£ 

£ 

u 



a 

rH 




£ 

a 

<4-1 



£ 

CO 

o 



4J 

£ 




CO 

0 

3 



*H 


r-H 



nd 

u 

O. 



1 CO 

£ 

a 



00 £ 

£ 

3 



£ O 

£ 




O X 

CX 

W 



rH £ 

O 


H 





0) 





4J 





H 

£ 




£ 

a 


4-4 


> 

£ 


O 


£ 

nd 




O 

CO 


00 


G 

£ 


£ 



£ 


*H 


rH 

U 




£ 

u 


£ 


4-J 



4J 


*H 

pH 

£ 

rH 


00 

a 

o 

•H 


•H 

£ 

•H 

4-4 


nd 

£ 

CO 


s 

1 

£ 

CO 

rH 

“■o 

O 

O" 

•H 

£ 

t 

•u 

£ 

0 

H 

i 

Q 1 

U 

CO 

4-J 

3 

2 : oo 

4-4 

£ 

£ 

u 

< o 

1 

£ 

CO¬ 

3 

***’*«» rH 

o 

U 

CO 

(X 

ed 3 

u 

-H 

\ 

o 

O 3 

1 


«H CO 


cO 

4-J 

£ 

£ £ 

rH 

60 

£ 

CO 

H CO 

3 

O 4-1 

£ 

rH 

O rH 

Cl 

i—1 *H 


£ 

a- £ 

3 

CO 34 

U 

cx 

g a 

3 

C 1 

£ 

0 

£ 0 

S3 

CO H 

a 

*H 

4-J *H 





no 





a 





£ 





£ 





J-4 





HO 





rH 


£ 

Ph 

£ 

£ 


O 

4-J 

O 

£ 


*H 


*H 

O 


4-J 

> 

4-J 

X 


£ 

*H 

a 

£ 


*H 

■U 

£ 


B 

4-> 

CJ 

nd 

4-J 

CO 

•H 

£ 

£ 

£ 

*H 

£ 


O 


£ 

♦H 

00 

a 

£ 

£ 


£ 


U 

HO 

rH 

*H 

rH 

£ 

o 

£ 

AS 

£ 

rH 

£ 

*H 

»H 

•H 

•H 

£ 

4-1 

CO¬ 

4-J 

£ 


£ 

CO 

£ 

4-4 

rH 

£ 


£ 


£ 

4-J 

£ 

4-J 

£ 

O 

o 

> 

o 

O 


tx 

•H 

ex 

*H 

_ CO 


4J 


4-J 


£ 

•H 

£ 

a co 

a 

o 

■U 

o 

£ 4-J 

cu 

*H 

£ 

•H 

nd £ 

o 

4-J 

ex 

4-J 

£ nH 

•H 

g 

£ 

CJ 

O O 

PQ 

< 

tx: 

< 

CJ CX 







CO 









£ 






rH 









00 






rH 

3 


3 






£ 



cn 

CO 


£ 

o 


3 

3 





U 



cn rH 

rH 


G 

•H 


a 

3 





o 



3 m 

rH 

rH 


3 


3 

3 





4-J 



o 3 

£ 

£ 

rH 

3 


•3 

3 





CO 



4—1 00 

G 

£ 

£ 

a 



3 








3. -H 


*H 

G 

3 


rH 

A 





£ 



04 cn 

£ 

4-J 

•H 

O 


3 

3 





> £ 



a 

O 

£ 

4-J 

>3 


3 

3 





*H *H 



04 34 

*H 

U 

U 

3 


•H 

3 





4-J CO 



3 O 

rH 


o 

•H 


3 

3 





£ >> 



o 

00 

£ 

G 



o> cn 

> 





•H rH 



4-1 3 

£ 

> 


a 


3 rH 






G CX 



o o 

£ 

*H 

£ 

3 


rH 

3 





o C 



33 -H 

00 

4-» 

> 

3 


60 3 

3 





CO 



(3 -3 


G 

nH 

3 


3 CJ 

s 





CO £ 



a 

rH 

£ 

4-> 

1 


•H 1 

o 





£ *H 



33 3 

£ 

rH 

G 

60 


rH X 

iH 








O 3 

£ 

£ 

CO £ 

3 


P 






nd £ 



3 3 

•H CO 

CO 

rH rH 

O 


3 3 

£ 





£ O 



33 

4-J rH 


rH £ 

rH 


3 3 

♦H 





£ «H 



4. 

£ rH 

rH 

£ CO 



•H 3 






4-J 



O 3 

H £ 

£ 

G 

33 


33 rH 

CO 





£ £ 



U 

G 

£ 


3 


^ 3 

rH 





9 P 



00 3 

4-J 

o 

£ 4-J 

3 


60 3 

rH 





*H M 



3 3 

£ U 

*H 

O -H 


£ 

3 -H 

£ 





4-J O 



•H 33 

G £ 

4-J 

*H Vj 

1 

00 

•H 3 

G 





£ 4-4 



iH 3 

rH 

G 

rH £ 

3 

£ 

rH 3 






4-J £ 



a 3 

CQ. O 

£ 

00 P. 

3 

U 

P 60 

U 





CU *H 



3 

cx 


£ CO 

O 

o 

3 

*H 





£ 



o o 

r -H 

*H 

£ *H 

x: 

4-J 

3 O 

£ 





nd 4-» 



O 3 

£ rO 

nd 

00 nd 

to 

CO 

3 3 

HO 





£ O 







£ 






























£ 



U 












O 



£ 












•H 


— 

4-J 












4-J 

PH 

3 

rH 





PH 







£ 

4-J 

O ; 

*H 

£ 




4-J 

4J 

3 





! G 

*H 

•H ' 

4-1 

O 




•H 

u 

O 





*H 

> 

3 i 


•H 




> 

o 

s= 





! 4-i 

•H 

•H 1 

4J 

4-> 




•H 

co. 

333 





•H 

4-J 

-£> i 

£ 

£ 




4-J 

1 

4-1 iH 

3 




nd 

G 

•H 

£ 





G 

CN 

cn o 

3 




O 

£ 

A ' 

£ 

4-1 




£ 


riA 

O 

£ 

- 


0 

£ 

3 

O 

£ 




£ 

rH 

CO cn 

3. 

O 

H 

3 


£ 

•H 

C0 

£ 




£ 

£ 

3 3 


*H 

O 

O 

G 

O 

H 

£ 

U 




O 

CJ 

3 3 

r—4 

4-J 

Z 

■H 

•H 

G 

O 3 


£ 

rH 



G 

o 

33 


*H 

1 

3 

4-J 


JZ 'H 


4-4 

O 




u 

s 4-1 

rH £ 

nd 

P 

3 

O- rH 

1 3 

rH 

4-4 

H 



rH 

cx 

04= 

£ G 

nd 


3 

£ 

£ 

p a 

£ 

*H 

4-J 



£ 

•H 

> 

G £ 

£ 

<o 

3 

£ 

£ 

Z 3 

G 

nd 

£ 



£ 

a 

•HTJ 

O £ 



. a. 


O 

<5 3 

•H 


O 



O 

£ 

4-1 *H 

U 4-J 


"W 

o - 

CO 

*H 

X 

H 

00 

G 



•H 

u 

3 o 

co. CO 

£ 

o 


1 

4-J 

rH 3 

4-J 

O Ph 




4-J 

1 

no a 

*H *H 

£ 

r-4 

o 

4-J 

G 

3 3 

G 

rH £ 

£ 



G 

£ 

04 00 

G CO 

£ 

3 

3 

CO 

£ 

a h 

£ 

£ -H 

*H 



£ 

o 

C3-H 

£ £ 

*H 

3 

3 

o 

£ 

o a 

rH 

£ £ 

£ 



£ 

£ 

r co 

u u 

rH 

3 

> 

3. 13 

rH Z 

£ 

£nd 

OO 



4-1 















‘ 3 













*H 


iH 













HO 


3 3 












1 

»H 


3 3 






1 



£ 



£ 

HO 


3 O 






3 



CO 



00 

£ 


33 3 




CO 


3 

G 


a 



£ 

•H 


s a 



G 

£ 

^ £3 

•H 


£ 



4-J 



3 



*H 

CO 

H 

£ 

4-J 


£ 



rH 

4-J 


a i—i 

a 


4-J 

cx 

O 

O 

•H 


rH 



o 

£ 


3 3 

•H 


co. 

£ 

4-J 

G 


CO 

CO 



> 

£ 


33 3 

4-J 


£ 

£ 

£ 


nd 

rH 







1 3 

a. 


£ 

Ph 

4-J 


£ 

£ 

X 



4-4 

M 


a 3 

£ 



CO 

*H 

U 

£ 

£ 

3 



o 

£ 


3 A 

£ 


CO 


G 

o 

nd 

£ 

Q 




G 


•< 3 

Ph 



pH 

X 

4-J 


£ 

3 

CO 


£ 

1 


3 1 

CO 


nd 

H 

£ 

*H 

£ 

HO 

*H 

£ 


O 

S 


+ 



£ 

O 


HO 

O 

G 

A 

£ 


*H 



<3 X 

nd 


4-J 

4-1 

£ 

•H 



•H 

£ 


4-J 



o 

£ 


£ 

£ 

£ 

HO 

£ 

0 

A 3 

U 


£ 

CO 


<3 

4-J 


•H 

4-> £ 

£ 

£ 

CO 

£ 

3 3 

HO 


rH 

rH 


X o 

£ 


nd 

•H £ 

£ 

*H 

o- 

•H 

•H -H 

0 


£ 

£ 


3 

*H 


£ 

G U 

4-J 


£ 

G 

a 

£ 


00 

£ 


•H 3 

nd 


0 

X 4-J 

£ 


£ 

rH 

33 cn 

0 


£ 

£ 


> o 

£ 


1 

£ 

HO 

00 


£ 

3 



£ 


•H *H 

0 

3 

pH £ 

G 


£ C0 

co 

G 

3 O 

£ 



HO 


3 3 

1 

o 

rH O 

nd *H 

£ 

•H 4-J 



•H 

> 



G 


•H 3 

>> iH 

rH »H 

£ 4-» 

O 

4-J £ 

>>3 

>> 3 

*H 


£ 



3 rH 

rH 

4-J 

£ U 

4-J -H 

*H 

£ a 

u 

4-J 

3 -H 

4-J 


4-J 

4-J 


3 X 

rH 

G 

G G 

£ A 

4-1 

£ £ 

o 


O 3 

G 


4-J 

£ 


3 3 

£ 

£ 

•H £ 

HO nd 

G 

HO «H 

4-J 

£ 

3 33 

£ 


•H 

£ 


cn O 

G 

nd 

U X3 

•H £ 

£ 

CO 

£ 


3 3 

l 


0 

nd 


A 

*H 

CO 

4-J CO 

U £ 

U 

V— ' (]J 

4-J 

£ 

3 3 

•H 


CO 

£ 


i a 

g 

£ 

G £ 

4-J nd 

£ 

G 

•H 

£ 

•H 33 

CO 


£ 

£ 

£ 

+ cn 

£ 

£ 

£ £ 

co 

4J 

nd £ 

G 

•H 

3 

£ 


£ 

a 

O 

CN o 

r£ 


rH M 

•H £ 

£ 

£ £ 

X 

to¬ 

X 3 

£ 


U 

£ 

*H 

3 A 

CJ 

4-J 

W 4-J 

Q *H 

M 

£ 4-J 

w 

co 

W O 

\ o* 

H 

nd 

4-J 

u a 


Long-distance action of neurotransmitter modulating and routing 

transmission of information 




















Koch and Poggio 


Biophysics of Computation 


1. Impulse Initiation and Conduction 


Action potential initiation and propagation were among the first biophysical mechanisms 
that were elucidated. Action potentials and their generation represent such a dominant 
feature of the nervous system, that for a considerable amount of time it was widely held that 
they underly all information processing in the nervous system. In the next four sections, we 
will provide a brief overview of their possible role in information processing. 


1.1 Impulse Initiation 
1.1.1 Biophysical Mechanism 

The first quantitative analysis of the currents underlying the action potential was done 
on the squid giant axon (Hodgkin and Huxley, 1952; for a summary, consult Katz, 1966; 
Hodgkin, 1964). The responsible agents are voltage-dependent Na+channels, described by 
two variables m and h. The inward current flowing through these channels is given by 


lNa(t) = g Na m z h(V(t) - E Na ), 

where V(t) is the voltage across the membrane, E Na is the reversal potential associated 
with the sodium ions and g Na is a constant. The activation variable, m, increases with 
increasing depolarization, while h, the inactivation variable, decreases. When the membrane 
is depolarized, the Na + conductance begins to increase, depolarizing the membrane further, 
and so on. This cycle proceeds until the membrane reaches the potential at which there is 
no more driving potential for the sodium ions, i.e. V = E Na , and until voltage-dependent 
inactivation sets in. In the meantime a counteracting, outward current, mediated by the 
outflow of potassium, is activated. It it usually described by 


IkM-V^VM-Ek), 


where n is the corresponding activation variable. Concurrent with the activation of the 
outward potassium current, the inward sodium current begins to inactivate, reducing the 
voltage to levels slightly below the initial resting level (see figure 1). Action potentials are 
all-or-none events, and therefore highly non-linear. Below threshold, a stimulus elicits only 
a local, graded response; above threshold, the membrane goes through its stereotyped 
voltage change independent of the stimulus intensity. Action potentials are all-pervasive, 
occuring in boih vertebrates and invertebrates (Hagiwara, 1983). 
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a) 



b) 



Figure 1. Typical axonal sodium (a) and dendritic calcium (b) action potential. Axonal spikes, 
in this case from the squid axon, display amplitudes of about 100mF and have a pulse width of 
about lms, Dendritic spikes typically have lower amplitudes and are more "smeared out". Shown 
is a Purkinje-cell dendritic spike following blockage of sodium current with TTX. Note the humps in 
the curve that suggest the presence of multiple sites for spike initiation. Adapted from Llinas, 1979. 

Calcium currents are also known, to trigger all-or-none electrical events (Kleinhaus and 
Prichard, 1975; Schwartzkroin and Slawsky, 1977; Wong, Prince and Basbaum, 1979; Llinas 
and Sugimori, 1980; Llinas and Yarom, 1981; Jahnsen and Llinas, 1984). Since most 
Ca-channels do not inactivate readily, their time course is usually longer than the time 
course of sodium spikes. Moreover, while the sodium conductances giving rise to spikes 
are predominately found in cell bodies and axons, high-threshold Ca 2+ spikes are believed 
to originate in the dendritic trees of neurons of the vertebrate CNS. 


1.1.2 Neuronal Operation 


The basic operation implemented by such a threshold mechanism is a digital OR or AND 
gate. If the threshold is low enough, a single electrical input will be able to surpass the 
threshold and trigger a spike. For larger threshold values, two or more converging inputs 
may be required to elicit a spike. The dependance of this scheme on the value of the 
threshold seems important in light of the observation that the threshold for eliciting spikes 
is known to depend on the prior spiking history. Thus, the firing threshold in frog sciatic 
nerve fibers and in cat motoneurons can increase by a factor of two for prolonged spiking 
activity (Raymond, 1979; Schwindt and Crill, 1982). 

The threshold scheme underlies a second operation: converting an analog graded signal, 
intracellular current or membrane potential, into a digital event, the spike. It is thus similar 
to a 1-bit analog-to-digital converter. 
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1.1.3 An Example of a Computation 

The early and influential view of information processing in neurons was expressed well in the 
threshold neuron model of McCulloch and Pitts (1943). In the most common version of this 
model, the neuron is considered as the elementary processing unit that receives excitatory 
and inhibitory inputs leading to depolarizing and hyperpolarizing dendritic potentials. These 
are subsequently propagated to the soma where they contribute—in a linear fashion—to the 
somatic potential. If the somatic potential exceeds a given threshold, a spike is initiated and 
transmitted along the axon. Otherwise the neuron remains silent. It is fairly straightforward 
to show that a set of these threshold-neurons is complete , that is every input-output 
behavior can be reproduced by appropriate combinations of threshold-neurons (given a 
fixed input-output coding in some fixed alphabet, like {0,1}; Kleene, 1956; Palm, 1982). 

1.2 Repetitive Spiking Activity 
1.2.1 Biophysical Mechanism 


One distinguishing characteristic of different types of neurons is their response to prolonged 
current injections or synaptic input. Responses range from a single burst to tonic, continuous 
spiking which can increase linearly or non-linearly with increasing stimulus amplitude, if the 
average firing frequency in response to a constant-current pulse in cat spinal motoneurons 
is plotted against the injected current strength, the resulting plot, the / -1 curve, is best 
described by two linear segments (figure 2). The steeper linear segment has been termed 
secondary range and the shallower segment the primary range (Kernell, 1970; Schwindt 
and Crill, 1982). For cat extraocular motoneurons and frog lumbar motoneurons, the / -/ 
curve is best described by only one linear segment. 

The underlying biophysical mechanisms are a slow, outward, calcium-dependent potassium 
current I I<S , a persistent calcium current 7, and the accommodative properties of the axon 
initial segment which lead to a rise in firing threshold with increasing firing rate (Schwindt 
and Crill, 1982; Crill and Schwindt, 1983). The potassium current is activated at 10mV 
or greater (relative to the resting potential E rest of the cell) and is partially responsible 
for the long afterhyperpolarization present in motoneurons, h is a steady, inward current 
activated at voltages traversed by the cell during steady rhythmic firing, showing little 
evidence of voltage-dependent inactivation. Depolarizing the membrane activates adding 
a substantial inward current and increasing the firing rate. Iks and the increase in firing 
threshold have the converse effect, reducing the / - I slope. Thus, the relative strengths 
of the tonic inward current and the outward current are the major determinant of the / - I 
relationship. 
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Figure 2. Average steady-state impulse frequency (/) plotted against intensity of injected current 
(/) in a spinal a motoneuron. There is an approximately linear relation between I and / over a range 
of lower and higher discharge frequencies (termed ’’primary" and "secondary" range). Adapted from 
Kernel, 1970. 


It is easy to imagine a calcium current which is already activated near the resting potential 
and which inactivates for depolarization. Injecting more and more current into the cell 
causes a "compensatory" decrease in /, through voltage-dependent inactivation of the 
calcium currents. The net effect may be a constant firing frequency, relatively independent 
of the injected current. Calcium channels in egg cell membrane do inactivate for increasing 
levels of membrane depolarization (Fox, 1981). The inactivation kinetics of the Ca? + current 
in this case is similar to, but slower, than the kinetics of the Na + channel, 

1,2.2 Neuronal Operation 


A single action potential carries only 1 bit of information, if its timing is irrelevant. Information 
is most likely coded in terms of the interval between succesive action potentials. In many 
instances the nervous system might well use the average spiking frequency as a convenient 
measure of information. The / - 1 relationship discussed above would then serve as 
current-to-frequency transducer , converting one sort of graded signal-, current, into another, 
spike frequency. Note that at this level of description, the code used is no longer digital, 
but analog. The slope of the f -1 relationship is primarily governed by two currents, Iks 
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and U. Reducing or enhancing their amplitude, by applying for instance a neurotransmitter 
or a neuropeptide to the cell, may change the slope of the input-output relationship, 

1.3 Impulse Conduction 

1.3.1 Biophysical Mechanism 


/'“'N 




In a cable with a sufficient number of voltage dependent Na + and K + channels, electrical 
impulses can propagate in a non-decremental fashion. Once an action potential has 
been initiated, the initial sodium current will spread to neighboring membrane patches, 
depolarizing the membrane and going through exactly the same cycle, and so on, down the 
length of the axon (Katz, 1966; Jack, Noble and Tsien, 1975). Since the shape of the spike 
remains constant over the whole length of the axon, the conduction is non-decremental. 
There are two different groups of spike-conducting fibers, myelinated and unmyelinated 
fibers. 

In myelinated axons, only found in vertebrates, conduction does not proceed continuously 
along the fiber, but jumps, as it were, from node to node. The channels responsible for 
the regenerative action potential concentrate at these small specialized sites (called nodes 
of Ranvier). This is known as saltatory conduction. The space between the nodes, the 
internodai segment, is covered by several hundred layers of myelin, what amounts to a 
high-resistance and low-capacity insulating substance (see, however, Funch and Faber, 
1984). While the two dominant currents for non-myelinated axons are sodium and potassium 
carried, the impulse at the mammalian node of Ranvier is generated almost exclusively by 
sodium currents (Chiu, Ritchie, Rogart and Stagg, 1979). Potassium channels may however 
be present below the internodai axonal membrane (Chiu and Ritchie, 1980, 1984; Kocsis 
and Waxman, 1980), Because ionic currents flow only at the node in myelinated fibers, 
saltatory conduction is also favorable from a metabolic point of view. Less energy must 
be expended by the Na-K pump in restoring the Na + and K + concentration gradients, 
which tend to run down as a result of prolonged spike activity in small fibers. 

Since axons can be up to several tens of centimeters long, the total travel time of the 
electrical impulses is of crucial importance, in unmyelinated fibers, the conduction times 
of action potentials is proportional to the square root of the fiber diameter (figure 3). 
In myelinated fibers, the conduction times are considerably decreased, being directly 
proportional to the diameter (Rushton, 1951; Pickard, 1969; Ritchie, 1982). Propagation 
velocity also depends on the number of active channels. If their number drops below a 
critical value, non-decremental propagation ceases (Sabah and Leibovic, 1972). 

1.3.2 Neuronal Operation 
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Figure 3. Relation between conduction velocity of action potentials, v, and fiber diameter d for 
small myelinated and non-myelinated fibers. Adapted from Waxman and Bennett, 1972. 

The axon is the equivalent of the wire or interconnect between (and not so much within) 
silicon chips, connecting the different computational devices, neurons, with a high-speed 
pathway. Thus, its main function is the transmission of impulses, viz. the transmission of 
information. The reliability of transmission depends on a number of factors, most notable 
the spike frequency and the specific morphology of the axon (see next section). Axons 
can have an extensive fan-out, making contact with a multitude of postsynaptic targets, via 
highly branched axonal trees (for an example in the visual cortex of the cat see Gilbert and 
Wiesel, 1983). 


1.4 Axonal Trees 


The all-or-none nature of action potentials has led to the idea that the axon serves mainly as 
reliable transmission line, connecting two or more information processing devices. The axon 
itseif is assumed to lack any processing function. This view might have to be broadened with 
the findings that in regions of membrane and geometrical inhomogenities the transmission 
of action potentials depends on a number of parameters, such as the geometry of the 
axon and the previous spiking history (for instance Krnjevic and Miledi, 1959; Tauc and 
Hughes, 1963; Bittner. 1968; Chung, Raymond and Lettvin, 1970; Grossman, Parnas and 
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Spira, 1979a,b). 


1.4.1 Biophysical Mechanism 





It has been shown both theoretically and experimentally that action potentials may fail to 
successfully invade the daughter branches of a bifurcating axon. As the theoretical analysis 
of Goldstein and Rail (1974) pointed out, the single most important parameter upon which 
propagation depends is the geometric ratio , 


GR 


d\ /2 + 4 /2 

J/ 2 ’ 

“0 


where and d 2 are the diameters of the daughter branches and d 0 the diameter of the 
main axon. GR equals the impedance ratio for branches of semi-infinite length. The use 
of the geometrical ratio assumes that the specific membrane properties are the same in 
all branches. For GR — 1, the action potential does not ’’see" the branching (i.e. perfect 
impedance match), and it propagates without perturbation past the branch point. GR = 1 
essentially captures Rail’s equivalent tree concept (Rail, 1984). If GR < 1, the action 
potential behaves as if the axon tapers and it speeds up. The more interesting situation 
occurs if GR > 1, i.e. if the combined load of the daughter branches is larger than the load 
of the main branch. When 1 < GR < 10, the action potential is delayed at the branch point 
and a ’’reflexion potential" appears (Parnas and Segev, 1979). If GR — 10 (e.g. GR — 10 if 
a' x — d 2 — 4.6d 0 ), propagation into both branches fails simultaneously, since the electrical 
load of the daughter branches has increased beyond the point where the initiation of the 
action potential in the daughter branches can be supported. Parnas and Segev (1979) 
emphasize that for each constant geometric ratio, changes in the diameter ratio of the 
daughter branches never yield differential conduction into the daughter branches. 

In a study of conduction failure in a branching axon of the lobster, Grossman et al. (1979a) 
report that, although the geometrical ratio is close to one, conduction across the branch 
point fails at stimulation frequencies above 30 Hz. Moreover, the conduction block appeared 
first in the thicker daughter branch and only later in the thin branch, thus calling for 
an extension of the Goldstein and Rail (1974) model. By increasing the extracellular K + 
concentration, Grossman et al. could mimick the block of conduction, suggesting that a 
differential buildup of extracellular potassium can account for the blockage. For prolonged 
trains of action potentials, Na + accumulates faster in the small diameter daughter branch, 
triggering the electrogenic Na + /K + pump earlier than in the thick branch. Therefore, the 
extracellular K' [ accumulation will be reduced faster than for the thick branch, preventing 
the early onset of conduction block. A further factor known to affect the conduction of 
action potentials is the average interspike interval (Chung et al., 1970). 
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1.4.2 Neuronal Operation 

Little if anything is known about the possible relevance of action potential conduction 
failures for information processing. Although it seems possible to control the conduction 
of action potentials into different branches, such a mechanism may not be robust enough 
to be widely applied in the nervous system. One could even argue that conduction block 
is an undesirable property of the system. One possible physiological function could be 
the protection of the system from over-excitation by limiting the total number of spikes 
transmitted. One possible role of this phenomenon for information processing could be 
the following. Virtually all axons, like those in the motor neuron or the geniculo-cortical 
projection cells, branch frequently before forming their final synaptic processes. In these 
structures, filtering may occur, where the pattern of impulses in the parent axon diverges 
into different patterns in its daughter branches (Chung et al., 1970; I. Segev, personal 
communication). The filtering may be temporal , where the frequency of the impulses in the 
daughter branches is different from the parent axon. On the other hand, the filtering could 
be spatial, where temporal patterns are resolved into spatial ones, i.e. different daughter 
branches show different firing behavior. 

1.4.3 Example of a Computation 

One instance where such a mechanism might be used to perform a specific function is in 
the excitatory motor axon innervating the muscle used for opening the claw in the crayfish. 
This axon branches to different regions of the opener muscle, two of which are called 
superficial distal and superficial central fibers. When the axon is stimulated at a frequency 
of 1Hz or less, junctional potentials (jp) recorded from the distal fibers are up to 50 times 
larger than jp’s in central fibers. At 80 Hz, central jp’s are up to four times larger than those 
observed in distal fibers. Since equal amounts of depolarization produce equal amounts 
of tension in both fiber types, it follows that distal fibers produce almost all of the total 
muscle tension at low frequencies of stimulation. Central fibers add an increasingly greater 
contribution to the total muscle tension as the corresponding nerve terminals begin to 
receive acton potentials in response to higher firing frequencies. The mechanism underlying 
this behavior is thought to be the more efficient invasion of action potentials into distal 
terminals at low firing frequencies (Bittner, 1968). 
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2. Synapses 

Specialized sites of contact between neurons, called synapses, represent one of the major 
means of communicating information between ceils. In this respect they can be compared 
to the pins on silicon chips, through which information is relayed to and from the chip. 
There are, of course, several critical differences. Different from today’s LSI chips with 
at most approximately 100 pins (see Blodgett and Barbour, 1982), the dendritic tree of 
neurons may carry up to 10 5 (morphological) synapses. The synapses themselves not only 
transmit information from the presynaptic to the postsynaptic neuron, using a chemical or 
electrical transducing mechanism, but may already process information, i.e. perform some 
computation. In the following three sections, we will consider the possible role of chemical 
and electrical synapses in information processing. 

2.1 Chemical Synapses 

2.1.1 Biophysical Mechanism 

/ N We will not attempt to review the literature concerning the biophysical and molecular 

mechanisms underlying synaptic transmission and release (see Katz, 1966; for a good 
overview see Shepherd, 1979b). Suffice it to say that the release of a chemical substance, 
the neurotransmitter, which crosses the cleft between the pre- and postsynaptic membrane, 
is essentially controlled by the amount of depolarization of the presynaptic terminal. The 
higher the presynaptic depolarization, the stronger the postsynaptic response, which can 
be either hyper- or depolarization. The transduction between presynaptic and postsynaptic 
voltage, V pre and V poet is usually described as either an almost linear or a sigmoidal 
transformation (figure 4). 

The neurotransmitter diffusing across the synaptic cleft activates voltage-dependent channels 
in the postsynaptic membrane, resulting in a change in membrane conductance g[t) in 
series with the ionic battery E. The conductance change induces in turn a voltage change 
V pi> st(t) (relative to the resting potential of the cell, E reat ). The change in potential is given 
by the Volterra equation (see for instance Rail, 1967) 


V poti {t) = {g(t)(E- V posi (t))} * K n (t) 

where Ku{t) is the input impedance (Green function) at the postsynaptic location and 
* represents convolution. For stationary inputs, or when the input changes slowly in 
comparison with the membrane time constant, this equation reduces to 
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P ° Bt 1 + gkn(0) ’ 

where Ka( 0) is the steady-state real input impedance at location i (essentially, the 
dc-component of the complex Fourier transform of Ku(t)). For a hyperpolarizing synapse, 
where an increased level of presynaptic potential leads to an increase in the postsynaptic 
hyperpolarization, the sigmoid relation as illustrated in figure 4 will be inverted (Graubard 
et a/., 1980). 

The roll-off at large depolarizations is expected in part from the finite number of receptors 
in the postsynaptic membrane and partly from the postsynaptic saturation effect, i.e. when 
the change in voltage V poet approaches the synaptic reversal potential E (see for instance 
figure 12). Subsequent increases in the presynaptic voltage, that is in the conductance g(t), 
will fail to evoke larger potential responses. Synaptic saturation can be an important factor 
in limiting the total dynamic range of the system. For instance, the dynamic range in insect 
monopolar cells is 10 1 — 10 2 , in contrast to the range of the presynaptic photoreceptor, 10 5 
(Shaw, 1981). One way to avoid having synapses with a low dynamic range is to distribute 
the synapses in the postsynaptic dendritic tree, thus minimizing synaptic saturation and 
avoiding the early onset of postsynaptic saturation (see section 2.3.1). 

A postsynaptic response graded by the amount of presynaptic depolarization is a crucial 
property for many synapses. A synapse from an axon terminal is normally activated by an 
action potential invading the terminal, but a synapse from a dendrite may be activated by the 
graded depolarization of synaptic potentials within that dendrite (Graubard, 1978). In fact, 
graded synaptic transmission may even occur between spiking neurons: thus, neurons in 
the lobster stomatogastric ganglion, in addition to eliciting spike-evoked inhibitory potentials 
in the postsynaptic cell, also release functionally significant amounts of transmitter below 
the threshold for action potential (Graubard, Raper and Hartline, 1980 and 1983; see also 
Hinas, 1979). 

One way of characterizing chemical synapses is to measure the synaptic amplification, called 
sensitivity or dynamic gain , by recording the change in postsynaptic voltage in response 
to a small change in the presynaptic voltage, i.e. dV poet /dV pre (for a good exposition of 
this method see Shaw, 1981 or Llinas, 1979). The maximum value of the dynamic gain 
identifies the most effective operating range at a particular synapse. For the chemical 
synapse between the photoreceptors and the lamina monopolar cells in the dragonfly 
retina, the maximum dynamic gain is about 34 at presynaptic depolarizations of only 0.7™.^ 
(Laughlin, 1973). This system, functioning without action potentials, can optimally signal the 
potential evoked by a single photon in the photoreceptor. Estimations of the dynamic gain 
in a homoiogue system, the rod-to-bipolar synapse in the dogfish, give similar numbers (at 
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Figure 4. Input-Output relationship for a "typical" synapse from a spiking neuron, in this case 
the giant synapse in the squid stellate ganglion. The voltages are given with respect to the resting 
potential. The maximum value of the dynamic gain dV po , t /dV prr « 3. Adapted from Katz and Miledi, 
1967. _ 

least 50; Ashmore and Falk, 1976 and 1979). By contrast, values for impulse-transmitting, 
non-sensory synapses are considerably less: 0.3 at a lamprey central synapse (Martin and 
Ringham, 1975) or approximately 3 at the squid giant synapse (Katz and Miledi, 1967; see 
figure 4). Both cases are optimized at presynaptic depolarizations of 60 - 80mK, with little 
observable transmission taking place below 30?nVT This evidence seems to suggest that 
in both vertebrates and invertebrates, synapses may be specialized in transmitting either 
very small voltages with no obvious threshold, i.e. converting one type of an analog signal 
into another analog signal, or in transmitting unitary impulses in a noise-free manner, i.e. 
converting a digital signal into an analog signal. 

2.1.2 Neuronal Operation 


A chemical synapse converts V pre into V vost (via a chemical process) and corresponds to 
a non-reciprocal 2-port in the sense of Brayton and Moser (1964) or Oster, Perelson and 
Katchalsky (1971). It can be described as a two-port element, since current and voltage 
can be varied independently at two sites, pre- and postsynaptically. Chemical synapses 
are non-reciprocal, since the input and output variables cannot be exchanged. Synapses 
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decouple neuronal systems from each other, rather like operational amplifiers, since the 
postsynaptic current can be varied without any change in presynaptic current. Reciprocal 
synapses , like those found between mitral and granule cells in the olfactory bulb (Shepherd, 
1979a) or between bipolar and amacrine cells in the vertebrate retina (Dowling and Boycott, 
1966; Ellias and Stevens, 1972), may reintroduce reciprocity. Since V poet may well be 
larger in amplitude than V vre , chemical synapses may amplify. Moreover, synapses can 
mimick positive or negative resistances, depending on whether their postsynaptic action is 
excitatory or inhibitory (Poggio and Koch, 1984). 


2.2 Electrical Synapses 


2.2.1 Biophysical Mechanism 


Interaction between neurons mediated by electrical coupling is a common feature of the 
nervous system in both invertebrates and vertebrates, including mammals (Bennett, 1977; 
Korn and Faber, 1979). Effective electrical coupling between cells is achieved through 
specialized low-resistance connections—the gap or electrotonic junction. A gap junction 
consists of two closely apposed membranes of adjacent cells, with the two membranes 
separated by a 2 - 4 nm space or gap. The junction itself allows the passage of ions or 
small molecules (of roughly 300 - 1500 daltons). Usually, gap junctions can be represented 
by an electrical resistance, constant over a range of ±25 mV (Bennett, 1977). Note that 
due to different values of the input-resistances on the two sides of the junction, this does 
not necessarily imply symmetrical propagation of impulses. Gap junctions can be strongly 
unidirectional or rectifying. In their study of electrical transmission at the giant motor nerve 
synapse of the crayfish, Furshpan and Potter (1959) showed that with small potentials 
across the junction, conductance was rather symmetric, but with potentials exceeding a 
few millivolts it was markedly nonlinear and asymmetric (figure 5). It has been possible to 
demonstrate that junctional permeability can be reversibly modulated by intracellular pH 
(Spray, Harris and Bennett, 1981), voltage across the junction (Harris, Spray and Bennett, 
1983) and, possible of great functional consequence, by inhibitory synapses (Spira and 
Bennett, 1972). 

An important difference in comparison to chemically mediated synapses is the lack of a 
substantial delay between the ’’presynaptic" depolarization and the "postsynaptic" response, 
usually averaging at least 0.25raa at chemical synapses in warm-blooded animals. 


2.2.2 Neuronal Operation 
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Figure 5. Current-voltage relationship of an electrical-rectifying junction in the crayfish giant motor 
nerve synapse. Such a synapse mimicks a diode. Adapted from Furshpan and Potter, 1959. 

Electrical synapses correspond to reciprocal 1-ports in the sense of Brayton and Moser 
(1964). Varying the potential on one side of an electrical junction leads to a change in 
the potential at the other side of the junction. Many electrical synapses can be described 
over a large voltage range by a constant, positive resistance. Thus, different from chemical 
synapses, they serve to couple cells electrically with little delay. Some electrical synapses 
behave like a typical rectifier device, implementing the biological analog of a diode. 
Moreover, they serve to synchronize the firing of nearby cells. An instance of such an 
operation are cardiac muscle cells, which are electrically coupled in an electrical syncytium 
(Torre, 1976). 


2.2.3 An Example of a Computation 


In the vertebrate retina, cone photoreceptors of the same spectral sensitivity have been 
shown to be electrically coupled via non-rectifying gap junctions, as have rods and horizontal 
cells of teleosts and turtles (Copenhagen and Owen, 1976; Kaneko, 1976; Detwiler and 
Hodgkin, 1979; Attwell and Wilson, 1980; Torre and Owen, 1983). It can be shown that 
when the membrane impedance of the photoreceptors can be linearized to a satisfactory 
approximation, analytical solutions to the network equations can be found that describe 
the dynamics of the response of any given cell to a variety of one- or two-dimensional 
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stimuli (Torre, Owen and Sandini, 1983). Such an analysis seems of particular importance 
when one realizes that under conditions of dark adaptation, for instance at night, the rod 
response to the absorption of single photons lies well within the linear response range of 
the cell. As a result of their analysis, Torre et al. suggest, following an idea of Hodgkin, 
that one functional role of coupling is to enhance the detection of signals. In particular, 
the signal-to-noise ratio decreases for small spots of light, resulting for instance from the 
random capture of photons, while the ratio increases for diffuse or long narrow patterns 
of light. Moreover, the coupling may serve to avoid aliasing due to the discrete sampling 
lattice of the photoreceptors. The Shannon sampling theorem (Shannon and Weaver, 1949) 
requires the spatial frequencies in the image to be below a critical frequency, in order for 
perfect reconstruction to occur subsequent to the sampling process. If higher frequencies 
are present, aliasing occurs, degrading the effective spatial resolution. Since the coupling 
can be equated with low-pass filtering the image falling upon the retina prior to sampling, 
it may help to reduce aliasing (Poggio, Nishihara and Nielsen, 1982; Torre et al., 1983). 


2.3 Interaction between Synaptic inputs 

When two neighboring regions of a dendritic tree undergo simultaneous conductance 
changes — induced by synaptic inputs — the resulting postsynaptic potential is in general 
not the sum of the potentials generated by each synapse alone. The postsynaptic potential 
induced by one synapse will propagate to the other synaptic site and change the driving 
potential at that location, i.e. the difference between the present postsynaptic potential and 
the reversal potential of the synapse. Thus, a different amount of current flows as a result of 
the "interfering" synapse nearby, allowing for the possibility that synapses situated "close" 
to each other may interact in a highly nonlinear way. We will briefly examine two different 
cases; the synaptic interaction between synapses of the same sign and the interaction 
between excitatory and inhibitory synapses. 

2.3.1 Biophysical Mechanism 

Nonlinear saturation: Let us consider an excitatory synapse that modulates the 
conductance g to a particular ion with an associated reversal potential E > 0 (relative to 
E reet ). In the steady-state case the change in somatic potential is given by 

v = 3f<U0)E 
6 i + gku(o)’ 

where W,- 6 (0) is the steady-state transfer impedance between the location i and the soma. 
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For small values of g, V, is proportional to gk ia (0)E and for very large values of g, V e 
saturates at k ia (Q)E/ku{0). This value therefore represents — under the assumption of 
a passive membrane — the maximal somatic depolarization evoked by a single synapse. 
We will now assume that the same synaptic input is spread among N identical synapses 
(each one with a conductance change g/N), Each one of the synaptic sites has about the 
same input impedance k«, the same transfer resistance to the soma k ia and the same 
transfer resistance k ( j between two synaptic sites i and j, Taking account of the nonlinear 
interaction between the different synaptic inputs, it can be shown (Koch, Poggio and Torre, 
1982) that the combined change in somatic potential is 

T , gk ia E 

V 8 - J 

l + gK u 

where 3T« = [#« + (# -1 )k ij }!N (in other words, the more the synapses are decoupled, i.e. 
kij -> 0, the smaller K u ). In other words, spreading the same conductance change among 
N at least partially decoupled sites enhances the maximal evoked somatic depolarization 
by a factor ku/Ku (Koch et a/., 1982). 

Nonlinear interaction: Let us consider the nonlinear interaction in a passive dendritic tree 
between an excitatory synapse at location e, with associated conductance change g e (t) and 
reversal potential E and an inhibitory synapse at i that increases the membrane conductance 
by gi[t.) to an ionic species having an equilibrium potential close to the resting potential of 
the cell, I — V reat — 0 (Pogio and Torre, 1978; Torre and Poggio, 1978). 'y-amino-butyric 
acid (GABA), acting via the classical bicuculline sensitive GABA a postsynaptic receptor 
and increasing a Cl~ conductance, would be an instance of a shunting inhibition. Activating 
shunting inhibition is similar to opening a hole in the membrane: its effect is only noticed if 
excitatory input is present. The resulting somatic potential is given by a system of Volterra 
integral equations: 


V.(t) = {g c (t)(E ~ V,(t))} * K ea (t) - {g,-(t)V,(i)} * K ia (t) 

V e (t ) - {ge(t)(E - V e (t))} * K"(t) - {g t (W(t)} * K ie (t) 

Vi(t) = {g e {t){E - V B (t))) * E ei (t) - { gi (t)Vi{t)} * K«(t), 

where AT,y(t) is the Green function of the system, i.e. the voltage response in time at location 
j if a i^-pulse of current is injected at i. V*, V { and V c are the values of the membrane 
potential at the soma, at the excitatory and at the inhibitory synapse, respectively. A simple 
measure of the effectiveness of shunting inhibition is the ratio (/*’) between the maximum 
of somatic depolarization in the absence of inhibition and in the presence of inhibition. 
For steady-state conductance inputs, it is possible to prove rigorously that for passive and 
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branched trees without loops the most effective location for inhibition is always on the 
direct path between the location of the excitatory synapse and the soma (Koch, 1982; Koch, 
Poggio and Torre, 1982). Rail observed earlier that in a single unbranched cable shunting 
inhibition effectively vetoes more distal excitation but not more proximal one (Rail, 1964). 

Detailed biophysical simulations of highly branched neurons (under the assumption of 
passive membrane properties) show that this on-the-path condition is quite specific: If the 
amplitude of the inhibitory conductance change is above a critical value (SOnS or larger), 
F values can be quite high (« 2 - 8) even when the excitatory inputs are much larger 
than the inhibitory ones, as long as the inhibition is between the excitatory synapse and 
the soma (figure 6). Inhibition behind excitation or on a neighboring branch 10 or 20//m 
off the direct path, is ineffective in reducing excitation significantly. If the amplitude of the 
inhibitory conductance change is too small, the strength of the interaction depends mainly 
on the distance between the two sites, and F values are very low. When the inhibition is not 
of the shunting type, i.e. I < V TCSt , inhibition at the soma is consistently more efficient (for 
more details see Koch et al., 1982). This specificity carries over into the temporal domain. 
Shunting inhibition must be activated shortly preceding or following the onset of excitation, 
in order to veto excitation effectively (figure 6; Koch, Poggio and Torre, 1983; Segev and 
Parnas, 1983). in summary, the interaction between excitation and shunting inhibition can 
be (i) strong, (ii) specific with respect to the relative position of excitation and inhibition 
and (iii) tuned to their timing. 

Our results hold also for the more unusual case of an input that decreases the membrane 
conductance for ions in equilibrium near the resting potential (see for instance Gerschenfeld 
and Paupardin-Trisch (1974), who provide evidence for a K + decreasing conductance in 
a molluscan neuron). In this case, the synaptic input facilitates the excitatory effect, thus 
implementing an analog approximation of a logical AND operation instead of the AND-NOT 
discussed previously (Koch et al., 1983). 

2.3.2 Neuronal Operation 

One operation implemented by synapses impinging upon the dendritic tree is a form 
of (nonlinear) addition. If the conductance change g is small in comparison with the 
input impedance A'„ at that location {gk« < l), then the conductance change g can be 
approximated as current gE and little interaction takes place between different synaptic 
inputs. This situation may easily occur if the synapse is located onto the soma or a primary 
dendrite with their low input impedance k it . In these cases the addition will be linear. If 
the induced conductance change is large, synaptic saturation must be taken into account, 
and the resulting somatic potential is less than the sum of the individual contributions. The 
linear, non saturating, operating range can be enhanced, however, by spreading the input 
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Figure 6. (a) A cat retinal ganglion cell of the 8 type (Boycott and Wassle, 1974). (b) The 
depolarization in the soma of the 6 cell for an excitatory conductance input g e at location 1 and 
an inhibitory conductance input g t at location 2. Both inputs have a rise-time of 25ms and decay 
to zero after about 90 ms. Location and timing of inhibition are optimal (i.e. the inhibition is delayed 
by 2.5ms). The excitatory battery is E = 80m T and the inhibitory battery / = OmV (relative to the 
resting potential). The corresponding somatic depolarization in the absence (V E ) and in the presence 
(Vr:~i) of inhibition also shown. Inhibition alone is "invisible" (because I = 0); its effect appears 
only when simultaneous excitation takes place, as expected from a multiplication-like interaction, 
(c) F factor (ratio of the maximum of the somatic depolarization without inhibition to the somatic 
depolarization with inhibition) for various locations of excitation and inhibition (as indicated in a) as 
a function of the relative timing. From Koch et at., 1983. 
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among several, at least partially decoupled sites, such as spines (see section 3). Under 
these conditions, nonlinear saturation will be substantially reduced. Such linear addition 
most likely occurs in the morphological a and p cell-classes of the cat retina, corresponding 
to Y- and X-cells, but not in the -7 and 6 cells (by exclusion identified with W-ce!ls; Boycott 
and Wassle, 1974; Koch et al., 1982). It may also be used by bipolar cells to maintain a 
large dynamic range despite the high synaptic amplification at the photoreceptor synapse 
(Torre and Poggio, pers. communication). 

The nonlinear interaction between an excitatory synapse and a shunting inhibitory synapse 
implements an analog veto operation, functionally equivalent to a logical AND-NOT gate . 1 
Because of the strength and specificity of the veto operation, it may perform characteristic 
information-processing operations in passive dendritic trees. Since inhibition vetoes more 
distal excitatory inputs only when it is on the direct path to the soma, a variety of local 
operations can be synthesized, exploiting the topology of the dendritic tree. Figure 7 shows 
a highly idealized picture of such ’’logical" operations implemented within a dendritic tree. 
Clearly, only certain types of logical operations can be carried out in a given dendritic 
topology. A cautionary note is in order here: this representation is primarily a convenient 
way of emphasizing some analogies between neuronal and logical information processing, 
while neglecting their differences. 

This behavior contrasts with the behavior obtained with a hyperpolarizing inhibitory synapse, 
such as GABA acting onto the bicuculine resistance, baclofen sensitive receptor controlling 
potassium channels (GABA b receptor; Newberry and Nicoll, 1384). In this case, the 
interaction between excitation and inhibition will be linear, that is the inhibitory synapse 
will reduce the EPSP generated by the excitatory synapse by an amount proportional to 
the inhibitory conductance change, eventually hyperpolarizing the cell for large enough 
inhibitory inputs. 

In the more unusual case of the postsynaptic conductance decreasing in response to the 
neurotransmitter, this mechanism implements an analog form of a digital AND gate (for 
instance Schulman and Weight, 1976). 

2.3.3 Three Examples of Computations 

Computing the direction of motion in retinal cells: Many neurons in both the retina 
and the visual cortex, exhibit the property of direction selectivity; they respond maximally 
to stimuli moving in a particular, the preferred, direction and minimally to stimuli moving in 
the opposite, or null, direction (Hassenstein and Reichardt, 1956; Hubei and Wiesel, 1959; 
Barlow and Levick, 1965; Oyster and Barlow, 1967; Cleland and Levick, 1974; Marchiafava, 


‘Note that AND-NOT is different from NAND. 
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Figure 7. (a) An idealized dendrite of a cat retinal ganglion cell of the 7 type (Boycott 

and Wassle, 1974), receiving excitatory (circles) and inhibitory (rectangles) inputs of the shunting 
type. Any inhibitory input can effectively veto only more distal excitation and does not affect 
other inputs more proximal to the soma. If the veto operation is described by an analog 
form of an digital AND-NOT gate, the operation "implemented" by the cell reads as [r 3 AND - 
NOT (»! OR i 2 OR t 3 )] OR [e s AND-NOT (», OR t 2 ) OR (e x AND-NOT » x )]. 

(b) An idealized dendrite of a i cell (see also figure 6 a) with excitatory and inhibitory 
synapses of the shunting type. Each inhibitory input (n - i 6 ) vetoes specifically only the 
corresponding excitation (e x - e 6 ) because it satisfies the on-the-path condition. The operation 
implemented by this cell is («1 AND-NOT * x ) OR (e 2 AND-NOT i 2 ) OR {[(e 3 AND- 
NOT i 3 ) OR (e 4 AND-NOT *„) OR (e t AND-NOT i 6 ) OR (e 6 AND-NOT t 8 )] 

AND-NOT i 7 }. From Koch et a/., 1982. 

1979; Jensen and DeVoe, 1983). Preferred and null directions are in general not predictable 

from the map of the receptive field nor can they be inferred from the global dendritic 

morphology of the direction selective ceil (Amthor, Oyster and Takahashi, 1984). Instead, 
direction selectivity has been postulated to be generated by asymmetries in the ganglion 
cell synaptic input (Torre and Poggio, 1978; Koch, Poggio and Torre, 1982). 


The seminal work of Barlow and Levick (1965) implied that inhibition is crucial for direction 
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Figure 8. (a) A part of the model of movement detection of Hassenstein and Reinhardt (1956). The 
two inputs are multiplied after low pass filtering with different time constants. If an average operation 
is made on the output the overall operation is equivalent to cross-correlation of the two inputs, (b) 
The scheme proposed by Barlow and Levick (1965) to account for direction selectivity of ganglion 
cells in the rabbit retina. A pure delay At is riot really necessary: a low pass filtering operation is 
sufficient (inhibition needs only to last longer than excitation), (c) The equivalent electrical circuit of 
the synaptic interaction assumed to underly direction selectivity as proposed by Torre and Poggio 
(1978). The interaction implemented by the circuit is of the type g i - ag,g 2 . The ionic battery of the 
inhibitory input g 2 is assumed to be near the resting potential of the cell. From Torre and Poggio, 
1978. 


selectivity: for motion in the null direction excitatory inputs are vetoed by inhibitory inputs 
originating in adjacent regions of the dendritic tree. As shown by Barlow and Levick 
(figure 8b; see also Wyatt and Daw, 1975), this veto operation must take place within 
small independent subunits contained within the receptive field which are extensively 
replicated. Torre and Poggio (see also Torre and Poggio, 1981) proposed that the nonlinear 
postsynaptic interaction between excitation and shunting inhibition underlies this behavior. 
Direction selectivity is achieved by asymmetric delay (and/or low pass properties) in the 
excitatory and inhibitory channels from the photoreceptors to the ganglion cell. Since 
nonlinearity of interaction is an essential requirement of this scheme, Torre and Poggio 
suggested that the optimal location for excitation and inhibition is on distal fine dendrites 
where the input impedance k n is expected to be high (figure 8c). 

Among the specific schemes that have been proposed to account for direction selectivity 
in the retina, two main classes of models have survived so far the experimental evidence, 
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Figure 9. Camera lucida drawing of an HRP-injected on-off direction-selective cell in the visual 
streak of the rabbit retina. The dendritic fields have been drawn in two parts: "outer" refers to the 
dendritic layer closest to the inner nuclear layer, while "inner" is the layer closest to the ganglion 
cell layer. Adapted from Amthor et at., 1984. 

based respectively on postsynaptic and on presynaptic inhibition. We briefly review these 
models and list their critical predictions. 

Postsynaptic models 

Detailed biophysical simulations of cat retinal ganglion cells with their highly branched 
dendritic trees, suggests that the AND-NOT mechanism is compatible with the available 
data about direction selectivity (Koch, Poggio and Torre, 1982 and 1983; Mistier, Amthor 
and Koch, 1985). In particular, intracellular recordings from directional selective ganglion 
ceils supports activation of a shunting inhibition in the null direction (Marcniafava, 1979; 
Ariel and Daw, 1982; Amthor, Oyster and Takahashi, 1983b). This inhibition effectively and 
specifically shunts excitation as long as it is localized near excitation or between excitation 
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and the soma. Since such a precise mapping (figure 7) imposes stringent requirement onto 
the specificity of the positioning of synapses during development, one particular simple rule 
to follow is that a pair of excitatory and inhibitory inputs always contact the ganglion cell 
close to each other. Computer simulations show that if excitation and inhibition are located 
very close to each other (i.e. within a few pm), inhibition will effectively veto excitation (for 
g e and <?,■ -> oo, F -+ gi/g e + 1; Koch et al., 1982). 

The direction of stimulus motion may thus be computed at many sites in the dendritic tree, 
independent of the threshold mechanism in the soma. On the basis of its nonlinear behavior 
and its highly branched dendritic architecture, Koch et al. (1982) conjectured that a 5-like 
cell morphology (in the cat Boycott and Wassle, 1974) represents an ideal substratum for 
direction selectivity (figure 6a). 

Recently, Amthor and his colleagues stained physiologically identified on-off direction- 
selective cells in the rabbit retina with intracellular injections of HRP (Amthor, Oyster and 
Takahashi, 1983a; Amthor et al., 1984). Their succesfully stained cells always show a 
bistratified, quite complex, dendritic tree (figure 9). These cells have extremely thin dendrites 
which carry spines or spine-like appendages; a rather puzzling feature is the presence of 
apparent ’’loops" created by retrofiexive branching within the dendritic fields. These loops 
seem to be a unique feature of direction-selective c^Ns in the visual streak, for cells with 
such apparent loops have not been reported to occur in the less highly branched bistratified 
ganglion cells of the same type in the retinal periphery (Famiglietti, 1984. It is therefore 
doubtful that the loops are actually electrically closed structures. Amthor et al. cell’s appear 
to meet the requirement for numerous local non-linear interaction sites, as postulated by 
Torre and Poggio (1978) and Koch et al., (1983). These distributed AND-NOT-like gates 
would compute direction selectivity throughout the dendritic tree, prior to somatic integration 
and impulse generation. 

The excitatory input in this model is likely to derive from motion-sensitive cholinergic 
’’starburst” amacrine cells (see below) while inhibition could possibly be mediated by the 
GABA releasing population of transient on-off amacrine cells (Miller, Dacheux and Frumkes, 
1977; Miller, 1979, Dowling, 1979; Vaughn, Famiglietti, Barber, Saito, Roberts and Ribak, 
1981). 

Presynaptic models 

An alternative to the postsynaptic models for direction selectivity in the rabbit is the 
hypothesis that the necessary computations occurs prior to ganglion cells. Although Werblin 
(1970) failed to record directional selective responses in any neurons presynaptic to ganglion 
cells (see also Marchiafava, 1979), this possibility cannot be ruled out, especially if the large 
number of different amacrine cell classes is taken into consideration (for a recent report of 
direction selectivity in retinal bipolar cells see Criswell and DeVoe, 1984). Ideal candidates 
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are ’’starburst" amacrine cells, believed to provide the cholinergic input to direction selective 
cells (Masland, 1980; Famiglietti, 1981, 1983b; Masland, Mills and Cassidy, 1984), Masland 
and his colleagues showed that these cells release ACh transiently at the on- or off-set of 
light. Starburst amacrine cells (figure 10), thought to be nonspiking, have dendrites that 
are probably decoupled from each other and the soma (Miller and Bloomfield, 1983). Only 
the distalmost portion of the dendrites give rise to conventional chemical synaptic output, 
while the bipolar and amacrine cell input is distributed throughout the cell (Famiglietti, 
1983a). Since each dendrite may act essentially as an independent subunit, they could be 
the morphological basis of Barlow’s and Levick’s subunits (1965). Moreover, GABA, crucial 
for direction selectivity (Caldwell, Daw and Wyatt, 1978; Ariel and Daw, 1982), appears 
to modulate the release of acetylcholine (Massey and Neal, 1979), favoring the scheme 
whereby an excitatory (bipolar) input in conjunction with the GABAergic Input implements 
an AND-NOT-like gate at the level of the starburst amacrine cell dendrites, whose output 
would then be directional selective itself. One would then predict that the application of an 
ACh potentiator should have little effect on the directional selective response of the ganglion 
cell. Physostigmine, an acetylcholinesterase inhibitor, does however eliminate directional 
specificity, in favour of the postsynaptic model of Torre and Poggio (Ariel and Daw, 1982). A 
second class of presynaptic models postulates a hyperpolarizing inhibition vetoing excitation 
in the null direction. Due to the rectifying properties of the amacrine-ganglion cell synapse, 
this circuit is functionally also equivalent to an AND-NOT gate. Although both presynaptic 
models are superficially rather similar, they correspond to very different computations. 
Thus, it can be shown that for particular stimuli (for instance, sinus gratings of different 
frequencies), the two models behave very different (Grzywacz and Koch, in preparation). 

EM immunocytochemical localization of glutamic acid decarboxylase (GAD) positive synapses 
in rat retina indicates that there are GABAergic inputs to bipolar, amacrine and ganglion 
cell processes in descending order of frequency (Vaughn et al., 1981), a result which is 
consistent with both the pre- and the postsynaptic model. 

We emphasize that all models of direction-selectivity relying on the nonlinear interaction 
between excitation and inhibition to discriminate between different directions, require 
a precise positioning of the different types of synapses (Swindale, 1983). The partial 
electron-microscopy reconstruction of a cat a ganglion cell (Freed and Sterling, 1983), 
supports the notion that such precision is, at least for one type of excitatory input, biological 
feasible. 

Although direction selectivity in the retina will probably represent the first critical test for 
the AND-NOT scheme, the proposed mechanism might play a role in other neurons of the 
central nervous system. In fact, the nonlinear synaptic interaction, perhaps augmented by 
membrane nonlinearities such as dendritic spikes evoked by EPSP’s for a stimulus moving 
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Figure 10. A starburst amacrine cell of the rabbit retina, filled with the fluoroecsent dye Lucifer 
Yellow. These cells release ACh transiently at the on- or off-set of the light stimulus. The scale bar 
corresponds to lOOftm. Adapted from Tauchi and Masland, 1984. 

in the preferred direction, may also be responsible for computing the direction of motion in 
cortical neurons of the cat and primates as well as underlying orientation selectivity in some 
cortical neurons (Koch and Poggio, 1984; Emerson, Citron, Feileman and Kaas, 1984). 

Computing relative motion in the fly: It has been known for some time that the housefly 
can discriminate relative movement of an object and its background, even when the two 
have identical texture (Virsik and Reichardt, 1976). The ability to discriminate objects in front 
of a distant background in terms of the relative velocity induced by motion is probably of 
particular importance to fast flying insects that cannot rely on binocular vision. Combining 
behavioral and electrophysiological experiments, Reichardt and his colleagues analyzed the 
optomotor response as a function of the spatial extent of motion and the velocity, contrast 
and spatial structure of the image. They proposed the basic structure of a neuronal circuitry 
underlying the detection of motion discontinuities, relating it to known properties of ceils 
in the third optic ganglion of the flies (Reichardt and Poggio, 1979; Reichardt, Poggio and 
Hausen, 1983). A critical component of the model is the presynaptic action of shunting 
inhibition on the output of the individual elementary movement detectors. This circuitry acts 
as gain control, making the optomotor response independent from the angular extent of 
the moving or oscillating figure, a well established phenomenon (Reichardt et a!., 1983). 
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Computing disparity in cortical cells: Studies in the alert monkey demonstrated that a 
large number of neurons in primary visual cortex (area VI) and an even higher proportion 
in prestriate cortex (area V2) are sensitive to horizontal disparities (Barlow, Blakemore and 
Pettigrew, 1967). Two main classes of stereoscopic neurons have been recognized (Poggio 
and Fischer, 1977; Poggio and Talbot, 1981): (1) cells that are selective for a very limited 
range of disparities, showing excitatory binocular facilitation (tuned excitatory neurons) or, 
less frequently, inhibitory interaction (tuned inhibitory neurons), and (2) cells that are tuned 
either to disparities corresponding to objects in front of the fixation plane (NEAR neurons), 
or those corresponding to objects behind the plane of fixation (FAR neurons). These early 
studies used isolated line and bar stimuli in the visual field. Thus, there is no possibility 
of making an incorrect correspondence between elements in the left and the right image. 
Poggio (1980, 1984) recently showed that roughly one fifth of all the cells whose depth 
sensitivity was examined responded to isolated lines as well as to random-dot patterns, 
in which abundant ambiguous matches occur. Interestingly, all of these cells were of the 
complex type. Thus, these neurons "solve" the correspondence problem of stereo (Poggio 
and Poggio, 1984). 

On the basis of the synaptic veto operation, we devised, together with Keith Nishihara, 
cellular models to account for the different types of disparity selectivity. Figure 11 shows 
two models for both excitatory and inhibitory neurons tuned for zero disparities and using 
edges as primitives (Poggio and Poggio, 1984). The synaptic input converging onto one 
dendritic branch and originating from the two eyes should have very similar receptive 
fields. The excitatory tuned disparity sensitive cell only responds to an edge if one eye 
signals ON activity and the other does not signal OFF activity for the same location, 
while the inhibitory cell always fires except if both eyes signal ON (or OFF) activity. The 
optimal disparity of these cells can be changed by varying the spatial offset between the 
receptive fields in the left and the right eye. The processing occurs in many independent 
subunits throughout the dendritic field of these neurons, using the AND NOT synaptic 
veto mechanism. This particular model seems consistent with Ferster’s "multiplication" data 
concerning the synergistic interaction between the two eyes (Ferster, 1981). Because ON 
activity in one eye vetoes the EPSP evoked by OFF activity' in the other eye, we predict that 
in an experiment using the pharmacological agent APB to block the ON channel (Schiller, 
1982), the excitatory tuned cell will lose its sensitivity to disparity. On the other hand, the 
inhibitory tuned disparity cell will continue to react to its tuned disparity in the presence of 
APB, although the signal-to-noise level will be decreased. Similar models, using AND NOT 
and AND-like "synaptic logic", can be devised for the NEAR and FAR disparity sensitive 


neurons. 
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Figure 11 . Schematic drawings of two cortical cells excited (a) and inhibited (c) by a given 
disparity. The excitatory tuned disparity sensitive cell responds only to an edge if one eye signals 
ON activity (clear circles) while the other eye does not signal OFF activity (hatched circles) for the 
same location, while the inhibitory tuned cell always fires except when both eyes signal ON (or 
OFF) activity. The disparity these cells optimal react to can be changed by varying the spatial offset 
between receptive fields in the left and right eye. (b) and (d) indicate the expected response of the 
cell to different disparities. 

3. Dendritic Spines 

Dendritic spines were originally discovered by Ramon y Cajal. Their existence was confirmed 
by Gray using electron-microscopy (Gray, 1959). A spine usually consists of a thin and 
slender spine neck and a more bulbous spine head (figure 12). Most important, every spine 
head has at least one synapse on its surface. These synapses are usually classified as 
Gray type I and are therefore believed to be excitatory. Most neurons in the CNS can be 
classified as either spinous or aspinous. In the former case, the majority of excitatory inputs 
can be seen to be localized on spines. Examples of this cell type are pyramidal and stellate 
cells in the cortex and purkinje cells in the cerebellum (Shepherd, 1979b). 

3.1.1 Biophysical Mechanism 


30 


tzcz a: 

ooo 



Koch and Poggio 


Biophysics of Computation 



Ijxm 

Figure 12. A schematic drawing of a typical dendritic spine in mammalian cortex with some of 
its constitutive elements. The postsynaptic density (PSD), directly below the presynaptic terminal, 
is a clump of electron-dense materia! made up of neurofilaments, actin, fodrin, tubulin, calmodulin, 
a microtubule-associated protein and other proteins. The spine-apparatus consists of two or three 
membrane-bound sacs, alternating with thin laminae of dense material. Also shown are the proposed 
voltage-dependent caicium channels, the sodium-calcium exchange pump and the ATP driven calcium 
pump. The scale is only approximate. From Robinson and Koch, 1984a. 

Dependance of the synaptic weight of a spine on its geometry: Since the synaptically 
injected current must pass through the relatively thin spine neck in order to reach the soma 
(Chang, 1952), the synaptic "weight" of the synapse is expected to depend strongly on the 
geometry of the spine. To study this effect Koch and Poggio (1983a,b) modeled dendritic 
spines using 1-dimensional cable theory. The spines are described by a thin cylinder of 
length l and diameter d and a thick and short spine head. The main assumption in this 
analysis is that the membrane is passive. Following the earlier studies of Rail (1974 and 
1978) it can be shown that the input resistance of a spine K n as seen by an imaginary 
electrode in the spine head is the sum of the input resistance in the dendrite just below 
the spine plus the resistance of the spine neck R N : 
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K 11 — K 22 + Rn- 

R n is simply the resistance of a cylinder, i.e. 4Ril/(nd 2 ). Thus, the spine input impedance 
depends strongly on the geometry of the spine neck. Since k n can be much higher than 
k 22 for small spine dimensions, current injected into the spine will produce a much larger 
local depolarization than if injected into the dendritic shaft. It turns out however, that it is 
irrelevant from the point of view of somatic depolarization, whether the current input is in 
the spine head or directly onto the dendritic stem, next to the spine base. This is easy to 
understand. Since the membrane surface of the spine is minute (« l pm*), essentially no 
current flows through the spine membrane (see also Koch and Poggio, 1984). Thus all the 
current injected into the spine head reaches the dendritic stem, or 

K u = k 2ei 
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where K ls (resp. K 2e ) is the transfer impedance between the spine head (resp. dendritic 
shaft below the spine) and the soma. 

Due to their expected high input impedance, spines can show large saturation effects (figure 

13). Saturation occurs not only for stationary or slow, but also for very fast conductance 

changes (Perkel, 1983; Koch and Poggio, 1983a). Figure 14 shows the dependance of the 

somatic potential on the geometry of the spine neck. If the amplitude of the synaptic input 

is small with respect to the inverse of the spine neck impedance (i.e. if Rn 9 < 1), synaptic 

input can be considered as current and little of interest occurs. If, however, the synaptic 

input is large (i.e. RNg > 1), significant saturation occurs, limiting the inflow of current into 

the spine. In this case, somatic depolarization induced by the synapse on the spine will 

depend strongly on spine morphology. The critical parameter determining the function of 

spines (for passive cable models) is thus the product of the spine neck resistance with 

the synaptic input rather then the spine input impedance (since both k n and k 22 can be 

0 

quite large but R N small; Turner, 1984). Unfortunately, little experimental data is available 
to bracket the size of these parameters. Very recently, Kawato, Hamaguchi, Murakami and 
Tsukahara (1984) estimated a peak conductance change of 43nS and a time-to-peak of 
0.3ms for somatic recorded EPSP’s in spiny rubrospinal neurons. Under these conditions, 
plausible changes in neck length could alter the "weight" of the synapse by a factor of 2 
or 3 (Koch and Poggio, 1983b). 

Until now we have only considered the case of a passive spine membrane. However, if the 
number of active, voltage-dependent channels in the spine head membrane is sufficient 
to trigger all-or-none impulses, the situation changes somewhat. Computer simulations 
show (Miller, Rail and Rinzel, 1985; Perkel and Perkel, 1985) that active, action potential 
generating membrane nonlinearities in the spine head can produce amplification rather 
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Figure 13. The maximum of the depolarization in the distal apical tree of a cortical pyramidal 
cell, for a fast transient conductance input of peak amplitude g (with a time-to-peak of 0.25ms and 
total duration of 0.9ms) of variable amplitude. The solid line indicates the potential at a synapse 
located witnm a dendritic spine while the dotted line shows the depolarization for a synapse located 
just below the spine, directly on the dendritic shaft. The dashed line shows that a steady-state 
conductance change yields essentially the same depolarization in the spine as the transient input. 
The membrane time constant is 8ms. The spine membrane is assumed to be passive. From Koch 
and Poggio, 1983a. 


than attenuation of the postsynaptic potential. The presence and amount of amplification 
depend on the density of active channels and on the spine-neck resistance. For a given 
spine head, there is an optimal range of spine neck resistances Rn which will lead to 
large EPSP’s at the base of the spine. Outside of this range, the spine acts as a voltage 
attenuator. 


Interaction between excitation and inhibition: A significant exception to the rule 
that spines carry only excitatory synapses are spines which show both symmetrical and 
asymmetrical synaptic profiles, i.e. excitation and inhibition, simultaneously. This situation 
occurs at about 5 to 15% of all cortical spines (Scheibel and Scheibel, 1968; Jones and 
Powell, 1969b; Peters and Kaiserman-Abramof, 1969 and 1970; Sloper and Powell, 1979; 
Somogyi, Kisvarday, Martin and Whitteridge, 1983) and seems to be the rule for the retinal 
input to X-like relay cells in the lateral geniculate nucleus in the cat (LGN; Famiglietti and 
Peters, 1972; Friedlander, Lin, Stanford and Sherman, 1981; Hamos, Raczkowski, van Horn 
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Figure 14. The somatic voltage corresponding to different neck dimensions for small, medium 
and large steady-state conductance inputs for a synapse located onto a spine in the distal apical 
tree of a cortical pyramidal cell. Fast, transient inputs yield very similar curves for peak conductance 
values, except that the somatic depolarization is scaled down by about a factor of 10. The spine 
neck dimensions were changed in such a way as to leave the total neck surface constant and equal 
to dl = 0.1/i7?i 2 . Plausible changes in neck length (for instance from 1.0 to 1.7 pm could alter the 
’’weight" of the synapse by a factor 2. From Koch and Poggio, 1983a. 


and Sherman, 1983; Hamori, Pasik, Pasik and Szentagothai, 1974). 


The nonlinear interaction between an inhibitory conductance change, with a reversal 
potential close to V rest , and an excitatory conductance change can be quite substantial and 
very specific, being essentially limited in their effect to the spine (Koch and Poggio, 1983a,b; 
Koch, 1984b). Thus, if the amplitude of the inhibitory input is large enough (g, > 0.05 10~ 6 S 
and gi > g e ), the depolarization evoked by the excitatory input is severely curtailed, dropping 
by a factor of 5 or more. Moreover, the potential at locations outside the spine are little 
affected by the inhibition. This is in accordance with the on-the-path principle (Koch et 
a!., 1982). When transient inputs are considered, the relative timing between them is an 
important determinant for the degree of interaction. Figure 15 shows the "tuning curve" of 
the F factor. As input functions we used fast transients with a time-to-peak of 0.25ms and 
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Figure 15. F factor (see figure 6c) for synapses on the spine head (continuous curve) or on the 
dendritic shaft just below the spine (dashed line) in the distal apical tree of a cortical pyramidal cell. 
Both input functions have a time-to-peak of 0.25m« and decay after 0.9ms. Inhibition is ten times 
stronger than excitation (see figure 6b); that is g imax = 2-10 ~ 7 S and g tmax = 10~ 8 S. Inhibition on a 
spine is more powerfull and specific than inhibition located on the dendritic shaft. From Koch and 
Poggio, 1983a. 

a total duration of about lm*. What is remarkable is that the vetoing effect of inhibition 
depends very sharply on relative timing beteen the two inputs. 'Whereas inhibition on the 
dendritic shaft can effectively veto excitation within a temporal window of the order of 
±0.3 ms, inhibition on the spine is stronger and more selective, being effective only in a 
window of ±0.12ms around the onset of excitation. 

Activity-dependent change in spine shape: It is well established that the form and 
shape of dendritic spines can vary following changes in environment and sensory input 
(see Coss and Globus, 1978 for a study in goldfish; Purpura, 1974 for a study of mentally 
retarded children; Bradley and Horn, 1979 for a study in chicken; Boycott, 1982 for a study 
in hibernating ground squirrels; Brandon and Coss, 1982 for a study in bees). Stimulating 
briefly the afferents to hippocampal cells results in hyperthrophy of the dendritic spines 
(Van Harreveid and Fifkova, 1975; Fifkova and Anderson, 1981; Fifkova, Anderson, Young 
and Van Harreveid, 1982). The hippocampus is of particular interest, since it displays a 
prominent modification of synaptic efficiency, termed long-term potentiation (LTP). 
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Free intracellular calcium is currently been regarded as one of the key factors underlying 
certains forms of synaptic plasticity (e.g. in Aplysia see Kandel, 1981; in mammalian 
hippocampus see Baimbridge and Miller, 1981; Turner, Baimbridge and Miller, 1982; Eccles, 
1983), Preventing a rise in [C'a 2+ ] i by injecting a calcium chelating agent such as EGTA 
(Lynch, Larson, Kelso, Barrionuevo and Schottler, 1983) blocks the establishment of LTP. 
It is of interest that the spine apparatus, a unique organelle localized in the spine neck and 
consisting out of two or more flattened sacs or cisternae, as well as the smooth endoplasmic 
reticulum inside the spine, has been shown to sequester Ca 2+ at high concentration 
(Fifkova, Markham and Delay, 1983). Once the concentration of intracellular Ca 2+ rises to 
substantial levels, the calcium can trigger a variety of mechanisms leading to a change in 
synaptic efficiency. 

One such mechanism, possibly implementing a transient change in synaptic efficiency, has 
been proposed by Crick (1982). If contractile proteins, such as myosin and actin, were to 
be localized in the spine neck, they could lead to a rapid contraction of the neck, possible 
within a fraction of a second. Functionally, this is equivalent to a rapid enhancement of 
the synaptic weight. Subsequently, different groups of researchers visualized actin in single 
neurons. While actin is present at all PSD, it concentrates in dendritic spines where it is 
either organized in long filaments oriented parallel to the axis of the spine neck or extends 
throughout the spine head in a lattice-like form (Katsumaru, Murakami and Tsukahara, 1982; 
Fifkova and Delay, 1982; Matus, Ackermann, Pehling, Byers and Fujiwara, 1983; Fifkova, 
Markham and Cullen-Dockstader, 1984). 

A second mechanism, possibly underlying long-term synaptic modification, is based on 
calcium-activated enzymes, calpain I and II, present in neuronal membranes (for a summary 
see Lynch and Baudry, 1984; Eccles, 1983). In the presence of free calcium, this protease 
breaks up a localized portion of the subceilular fodrin network, irreversibly uncovering 
glutamate receptors (Siman, Baudry and Lynch, 1983). Since glutamate, or a closely 
related amino acid, most likely serves as transmitter in several hippocampal pathways, this 
uncovering will increase the postsynaptic conductance change g{t) and thus the weight of 
the synapse. We would like to note, however, that for the unmasking of the receptors to 
increase the synaptic efficiency, two conditions need be meet. First, the factor limiting the 
size of the postsynaptic conductance change must be the limiting number of postsynaptic 
receptors and not the amount of transmitter released by the presynaptic terminal. Second, 
the conductance change prior to the unmasking must be small in relation to the input 
impedance of the spine. If significant synaptic saturation occurs, increasing g(t) will do little 
to increase the corresponding EPSP. 

On the basis of detailed computer simulations, Robinson and Koch (1984a,b) attempt to 
model the dynamics of free calcium in dendritic spines. They assume that presynaptic 


36 



Koch and Poggio 


Biophysics of Computation 


electrical activity induces, through opening of aspartate or glutamate channels, a large 
depolarization in the spine head. The EPSP in turn activates voltage-dependent Ca? + 
channels, resulting in an influx of calcium into the spine (Perkel and Brown, 1982). Such 
calcium channels are believed to exist in the dendritic arbours of in vitro purkinje cells 
(Hinas and Hess, 1976; Hinas and Sugimori, 1980), olivary neurons {Llinas and Yarom, 
1981) and in vitro thalamic neurons (Jahnsen and Llinas, 1984). The inflowing calcium 
will be rapidly bound by high-affinity calcium buffers such as calmodulin and calcineuron, 
both of which have been localized in substantial amounts in spine cytoplasm and PSD 
(Grab, Carlin and Siekevitz, 1980; Wood, Wallace, Whitaker and Cheung, 1980; Klee and 
Haiech, 1980). These buffers, together with the Na + -Ca 2+ exchange pump, the ATP-driven 
Co 2+ pump and the diffusion of calcium into the dendrites, keep [Ca 2+ ]; low in response 
to moderate presynaptic activity. If, however, the presynaptic activity exceeds a critical 
amount, the calcium buffers will saturate. Since the volume of a dendritic spine is small, 
very few presynaptic spikes will be sufficient to drive the level of [Ca 2+ ], from its resting 
value around 0.1 \iM to values between 10 and 500/^M. As Robinson and Koch specifically 
point out, the optimal strategy to saturate the buffers are very closely timed spikes, i.e. 
bursts. 

The free intracellular calcium can now activate calpain I or II, leading to the irreversible 
uncovering of the glutamate receptors. Another possibility is the binding of calcium to 
calmodulin. Once calmodulin is fully activated, for which all four calcium sites must be 
occupied, it induces actin-myosin contractions (Klee and Haiech, 1980; Cheung, 1982). The 
time course of these contractions — and thus of synaptic enhancement — is governed by 
the rate at which calcium becomes unbound from the calmodulin, which in turn depends 
on [Co 2+ ]j. A hypothetical time course of [6V+], and activated calmodulin for prolonged 
synaptic inputs is shown in figure 16. 

Calmodulin will also interact strongly with another major cytoskeleton protein, fodrin or 
brain spectrin (Kakiuchi and Sobue, 1983), responsible for giving the neuronal membrane 
its rigidity. The strong binding of calmodulin directly to fodrin might cause a change in the 
fodrin-actin network such that loosening of the spine cytoskeleton occurs. 

In brief, if calcium channels were to be found at the spine membrane, dendritic spines, due 
to their special geometry which limits the effects of diffusion and enhances local EPSP’s, 
may implement a calcium-dependent modification in synaptic efficiency spanning different 
time-scales. This mechanism itself is prone to modification, for instance through voltage, 
giving rise to possible Hebbian-like interactions between nearby spines. Cooperativity among 
afferents to CA1 cells in inducing and maintaining LTP has been reported (Lee, 1983). 


3.1.2 Neuronal Operation 
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Figure 16. A hypothetical time-course of free intracellular calcium and of activated calmodulin 
in a dendritic spine during prolonged presynaptic activity {in this case at 300 Hz for 120ms). The 
stimulus-induced influx of Ca~ + is buffered within a fraction of a millisecond. Each spike will, 
however, lead to a small increase in [Ca' jT ]i and thus increase the fraction of calcium bound to 
caimodulin (see inset). Once the presynaptic input ceases, both [Ca 2+ |,- and the activated calmodulin 
decay slowly, their time-course being limited by the low off-rate constants of the buffers (Klee and 
Haiech, 1980). A sudden increase in intracellular calcium during and following repetitive presynaptic 
stimulation in hippocampal neurons has been reoorted (Morris, Krnjevic and Ropert, 1983). Detailed 
computer simulations, describing the time-course of calcium and the various calcium buffers in a 
3-compartment model as a function of presynaptic activity, confirm such a behavior (Robinson and 
Koch, 1984b). 

Dendritic spines could be the morphological substrate for at least two different neuronal 
operations. 

First, they could serve as locus for modifying the functional connectivity between individual 
neurons, changing synaptic weights within a fraction of a second (via actin/myosin 
contractions) to several days or even weeks (via structural changes in the cytoskeleton). 
It is likely that this synaptic modification is triggered by an activity-dependent calcium 
influx. These mechanisms are crucially dependent on the potential buffering abilities of the 
spine cytoplasm. The calcium buffers essentially act to delay the onset of the synaptic 
enhancement and protect the system from noise induced by spontaneous presynaptic 
activity. The change in functional connectivity is primarily controlled by the amount and 
timing of the presynaptic input and secondarily by postsynaptic electrical activity. In 
particular, if the proposed calcium influx is mediated by voltage-dependent Ca 2+ -channels, 
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hyperpolarizing the spine would prevent these channels from opening, thus implementing 
a kind of Hebbian rule (Hebb, 1948; Palm, 1982). 

Second, the synthesis of a circuit consisting of an excitatory and an inhibitory synapse 
localized onto the same spine, may implement a localized AND-NOT gate. Inhibition vetos 
excitation very efficiently as long as inhibition is activated in a narrow temporal domain 
around the onset of excitation. Owing to the particular geometry of the spine, inhibition will 
do little to influence the electrical activity in the dendritic tree outside the spine. In other 
words, even though the actual veto operation occurs at a postsynaptic site, it is functionally 
equivalent to presynaptic inhibition. 

3.1.3 Two Examples of Computations 


/T-S. 



Information storage: One obvious function of modifiable spines is information storage 
over short and long time ranges (Rail, 1974, 1978; Jack, Noble and Tsien, 1975; Koch and 
Poggio, 1983a,b; Perkel, 1983; see especially Crick, 1982; Eccles, 1983; Lynch and Baudry, 
1984). Differing from current digital computers, where the memory resides in specialized 
hardware — usually kept separate from the logical processor — biological memory would 
be distributed throughout the information processina machinery, coded in form of the 
strength of connectivity between the different computational machines. One instance of an 
experimental well-known paradigm where information could well be "stored" in dendritic 
spines is LTP in the mammalian hippocampus (for an overview see Chung, 1977 and 
Swanson, Teyler and Thompson, 1982). It is at the moment still unclear, however, how the 
actual storage and retrieval of information could take place. 

Disabling visual input in the LGN: Recently, electron-microscopic studies of relay cells 
in the cat LGN have shown that the retinal input of X-cells is associated with a special 
synaptic circuitry, termed the spine-triad complex (Famiglietti and Peters, 1972; Hamos 
et ah, 1983, 1984; see also Wilson, Friedlander and Sherman, 1984). The retinal afferent 
makes an asymmetrical synapse with both a dendritic appendage of the X-cell and a 
geniculate interneuron. The interneuron contacts in turn the same dendritic appendage with 
a symmetrical synaptic profile. Recent evidence shows that these geniculate interneurons 
stain for glutamic acid decarboxylase (GAD), the synthesizing enzyme for GABA (Sterling 
and Davis, 1980; Fitzpatrick, Penny and Schmechel, 1984). The retinal input to geniculate 
Y-cells is predominately found on dendritic shafts with no triadic arrangement. On the basis 
of a computer model of an anatomically reconstructed X-relay cell with known somatic input 
resistance, Koch (1984b) showed that under the assumption that the geniculate interneuron 
mediates a shunting inhibition, activation of the interneuron by cortical or midbrain inputs 
can reduce very efficiently the excitatory postsynaptic potential induced by the retinal 
afferent without affecting the electrical activity in the rest of the cell. Since Y-cells lack the 
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spine-triad structure, inhibition acts globally, reducing and damping the general electrical 
activity of the cell. Anatomy and electrophysiology indicates that the geniculate interneurons 
receive an excitatory projection from the visual cortex (Jones and Powell, 1969a; Ahlsen, 
Grant and Lindstrom, 1982) and inhibition from both the midbrain (Singer, 1977; Foote, 
Mordes, Colby and Harrison, 1977; Ahlsen, Lindstrom and Lo, 1984) and the thalamic 
reticular nucleus (Guillery, 1969; Wilson et a/., 1984; Hamos ef a/., 1984). Therefore, Koch 
proposes that geniculate interneurons selectively gate the flow of visual information into the 
X-system as a function of the behavioral state of the animal, enhancing the center-surround 
antagonism and possibly mediating reciprocal lateral inhibition and eye-movement related 
suppression. Electrophysiological evidence in favor of such a dichotomy between the 
geniculate X- and Y-system in terms of the action of inhibition can be found in Singer and 
Bedworth, 1973; Noda, 1975; Tsumoto and Suzuki, 1976; Fukuda and Stone, 1976; Foote et 
ai., 1977; Derrington and Fuchs, 1979; Bullier and Norton, 1979 and Berardi and Morrone, 
1984. 
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4. Membrane Nonlinearities 

/"N 

Ever since Hodgkin and Huxiey identified a fast sodium and a delayed rectifying potassium 
current as the two major currents underlying the generation of action potentials, the total 
number of currents with widely different dependencies on voltage, time and chemical 
substances, has multiplied tremendously. What is the function of these currents for 
information processing? In the next three sections, we will focus on those currents which 
do not generate all-or-none impulses but modify the electrical behavior of the cell. 

4.1 Quasi-Active Membranes 

4.1.1 Biophysical Mechanism 

Usually, neuronal membranes are classified as either passive, i.e. the voltage decays 
exponentially to zero, or as active, like for instance the axonal membrane giving rise to 
action potentials. Between these two extremes, there is a range of membrane behavior we 
term quasi-active. Such a membrane still behaves linearly, i.e. doubling for instance the 
applied current doubles the voltage response, but it already shows oscillations. We define 

- a membrane to be quasi-active if it shows bandpass-like behavior, i.e. if the membrane 

impedance has a prominent maximum at some non-zero frequency, called the resonant 
frequency, f max . Such a membrane can always be modeled by an electrical circuit containing 
an inductance. 

What are the biophysical mechanisms leading to such a behavior? An electrical component 
can be described on a phenomenological level by an inductance if its voltage is proportional to 
the current change. As Detwiler, Hodgkin and McNaughton (1980) pointed out, this situation 
arises for a time- and voltage-dependent K + conductance activated by depolarization 
and inactivated by hyperpolarization. Examples are the currents mediated by potassium 
channels in the squid axon (Hodgkin and Huxley, 1952; Mauro, Conti, Dodge and Schor, 
1970), or by the calcium-dependent K + current in sympathetic neurons and hair cells 
of the bullfrog (Adams, Constanti, Brown and Clark, 1982; Lewis and Hudspeth, 1983). 
Alternatively, an inductance can mimick the small-signal behavior of the Na + (or Ca 2+ ) 
conductance inactivated by depolarization and activated by hyperpolarization as for instance 
the low-threshold calcium mediated current in mammalian CNS neurons (Llinas and Yarom, 
1981; Jahnsen and Llinas, 1984b). For small enough excursion of the voltage around a 
fixed potential V, these channels can be described by a circuit similar to the one shown in 
figure 19a. 

The presence of an inductance can lead to resonant behavior, i.e. the membrane impedance 
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Figure 17. The membrane impedance s m for two types of membranes as a function of frequency. 
Calculating the small-signal impedance of a patch of squid axon membrane described by the 
Hodgkin-Huxley equations gives rise to the bandpass function, which peaks at 67 Hz. Reducing the 
channel density ^ to zero, leaving only the passive components, yields the monotonic decaying 
curve. Note that beyond 200 Hz both curves coincide. From Koch, 1984a. 


shows a maximum at f max . A sinusoidal current of that frequency will therefore decay 
less than current with lower or higher frequency components, that is, the impedance 
will show bandpass-like character (figure 17). Thus, excitable membranes are capable of 
producing subthreshold oscillatory responses near 100 Hz when K + and Na + carry the 
current (Hodgkin and Huxley, 1952; Mauro et a/., 1970). The resonant frequency can be 
modulated by the density r? of active channels (Sabah and Leibovic, 1972; Koch, 1984a) 
and, perhaps less likely, by [Ca 2+ ],-. It can be as low as 1 Hz as in heart cell membrane 
(Clapham and DeFelice, 1982) or as high as several hundred Hz as in the frog node of 
Ranvier (Clapham and DeFelice, 1976). Increasing r? increases the degree of membrane 
excitability and the resonant frequency while decreasing the range of validity of the linear 
circuit approximation. 


4.1.2 Neuronal Operation 
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Membranes showing quasi-active behavior within a given voltage range can implement at 
least two different operations. First, such membranes may act as an electrical resonant 
filter , preferentially responding to inputs within a certain frequency range. The value of 
the resonant frequency can be varied by modulating some of the parameters underlying 
excitability, like the density of active channels or the level of free, intracellular calcium 
(Ashmore, 1983; Koch, 1984a). 

Second, quasi-active membranes may act as analog, temporal differentiators (Koch, 1984a). 
Intuitively, this can be understood by noticing that differentiation is nothing but a high-pass 
filtering, removing low frequencies. The differentiation of a function f(t) corresponds to the 
multiplication of the Fourier transformed function with frequency, i.e. 


df(t) 

dt 


—* iu> 


7(<4 


If the frequency components above the resonant frequency are disregarded, the highpass 
component of the membrane impedance can be approximated by 


/’’"N 


z m (u) = a + bu) 

with a and 6 positive constants and 6 > a. Injecting a current l(t) containing little or 
no frequency components above / mttX into a cable with a quasi-active membrane leads 
therefore to a change in potential 




V(t) = al(t) + b 

Figure 18 demonstrates such a case. The current I(t) (figure 18a) is injected into an infinite 
cylinder with a linearized Hodgkin-Huxley membrane (Hodgkin and Huxley, 1952). Figure 
18b shows the analytical derivative of I{t), while in figures 18c and 18d the "recorded" 
voltage change at two different locations in the cable is plotted. Such a situation may 
possibly occur for instance in the photoreceptor of the fly. Its output process onto a lamina 
monopolar neuron does not generate spikes under physiological conditions. The visual 
evoked response in the postsynaptic neuron to a flash or to a step increase of light can 
be well approximated by the scaled derivative of the response of the photoreceptor as 
recorded in its soma (Hengstenberg, 1982). 

A third operation possibly implemented by quasi-active membranes is spatio-temporal 
filtering in visual neurons (Detwiler et a!., 1978 and 1980; Koch, 1984a). 

4.1.3 One Example of a Computation 
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Figure 18. Injecting a subthreshold current, shown in (a), into an infinite cable with a linearized 
Hodgkin-Huxley membrane, (b) shows the analytic derivative of the injected current, (c) and (d) are 
plots of the resulting change in potential at the location of the current injection (c) and 350 pm 
farther away. While the current peaks at 5 ms, the recorded voltages reverses their polarity at 11.3 
and 10.5ms. Note the similarity between the derivative of the current and the recorded voltage. 
Adapted from Koch, 1984a. 


Individual sensory neurons, converting sound pressure or electrical fields into action 
potentials, have an optimal operating range in terms of the frequency of the input to which 
they are most sensitive. Such differential receptor tuning has been observed in hair cells 
of the cochlea in lower vertebrates (Ashmore, 1983; Crawford and Fettiplace, 1980 and 
1981a,b; Lewis and Hudspeth, 1983) or in fish electroreceptors thought to derive from hair 
cells (Hopkins, 1976; Meyer and Zakon, 1982). Hair cells, tonotopically organized along the 
basilar membrane in the cochlea according to their characteristic frequencies (the frequency 
of the sound stimuli the cell optimally responds to), reveal their electrical tuning by damped 
oscillations of the membrane potential induced by current injections. The frequency of the 
induced oscillations coincides with the characteristic frequency, in mammals the basis of 
frequency discrimination is believed to be the mechanical travelling wave set up in the inner 
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ear by the sound wave and first described by von Bekesey (1960). In some lower vertebrates 
such as the turtle or the bullfrog the tuning effect is believed to be an electrical resonance 
mechanism located in every hair cell (Crawford and Fettiplace, 1981a) and described by an 
circuit of the type shown in figure 19a (Crawford and Fettiplace, 1981b). The question of 
the ionic nature of the underlying mechanism has been investigated recently by Lewis and 
Hudspeth (1983) using the patch clamp whole-cell technique. They report the presence of 
three different types of channels (Ca 2+ -, K + - and calcium-dependent K + -conductance), 
one or several of them being responsible for the tuning effect. In order to explain a) the 
large range (70 - 700 Hz) of the characteristic frequency shown by hair cells and b) the 
systematic variation of this frequency along the basilar membrane, it has been proposed 
that the nature of the individual channels remains invariant for all cells while the density 
of channels varies from cell to cell (Crawford and Fettiplace, 1981a; Koc.i, 1984a). Due to 
this varying channel density and the concomittant varying degree of electrical excitability, 
Koch (1984a) predicted a correlation between the characteristic frequency of individual hair 
ceils and their degree of nonlinearity. While this mechanism leads to a fixed, stationary 
distribution of characteristic frequencies, transient effects might be achieved by a shift in 
the concentration of some ions iike calcium (see Ashmore, 1983), neurotransmitters or even 
hormones as demonstrated in electric fish (Meyer and Zakon, 1982). 

Hair cells seem like ideal candidates for computer simulations, since their role in information 
processing relies on non-spiking membrane mechanisms which can be modeled using a 
Hodgkin-Huxley like formalism (for a recent attempt see Lewis, 1984). 

4.2 Transmitter Regulation of Voltage Dependent Channels 


In the last years the basis of classifying conductances into those which depend only on 
voltage and those which depend only on the presence of a neurotransmitter (Grundfest, 
1957) has gradually been eroded. There are now several examples where a transmitter 
does not open new ionic channels but modifies a voltage-dependent channel. Examples of 
such mechanisms include the control of cardiac calcium current by /^-adrenergic agents 
(Reuter, Stevens, Tsien and Yellen, 1982), the action of opiates on neurons of the 
myenteric plexus and locus coeruieus (North and Williams, 1983), the synaptic inhibition of 
voltage-sensitive currents in Aplysia (Klein and Kandel, 1980) and the muscarinic control of a 
K + conductance in bullfrog sympathetic neurones (Adams and Brown, 1982; Adams, Brown 
and Constanti, 1982a,b) and in the mammalian hippocampus (Halliwell and Adams, 1982; 
for a general survey see Siegelbaum and Tsien, 1983). To illustrate this new development 
and its implications for neuronal computations we will use as a well-characterized example, 
M-current inhibition. 
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Figure 19. (a) Equivalent RLC circuit used to model the electrical resonance in turtle hair cells. 

(D) The sensitivity of a hair cell (voltage response normalized to the sound pressure of the tone 
used) as a function of the frequency of the test tone (P). The cell operates in its linear range. 
The frequency selectivity derives from a broad bandpass filter common to all hair cells (most likely 
originating in the middle ear; indicated by the dashed line) and an electrical circuit like the one 
show in (a) unique to every cell. Adapted from Crawford and Fettiplace, 1981b. 


4.2.1 Biophysical Mechanism 


Stimulation of cholinergic preganglionic fibers produces two very different excitatory 
responses in amphibian (B-type) sympathetic neurones: a very fast EPSP with a superimposed 
spike and a subsequent slow EPSP of small amplitude (figure 20a). The latter begins after 
a latency of some 200- 300ms, peaks at about 2s and lasts some 10 -20s (Adams and 
Brown, 1982; see also Brown, 1983). Both potentials result from the release of ACh from 
the same preganglionic fibers. The initial fast EPSP, analogous to the "classical" muscle 
end-plate potential, results from the action of ACh on curare-sensitive nicotinic receptors 
while the slow response is a result of ACh acting on a muscarinic receptor. Adams and his 
colleagues showed that the mechanism underlying the slow EPSP is a selective inhibition 
of the M-current (1 M ), a time- and voltage-dependent A' + -current (Adams et a/., 1982a,b). 
I.m shows no detectable inactivation. It turns on rather slowly, starting at around -60mF 
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and is maximally activated near -20 mV (Brown and Adams, 1980). Because it shows no 
inactivation, it strongly influences the steady-state I- V relation near the resting potential. 
Under normal conditions the slow EPSP generated by a single preganglionic stimulus closes 
some 5 -10% of the open M-channels; with repeated trains of stimuli up to 50% of the 
channels can be closed. Im inhibition usually produces a striking increase in the excitability 
of the neuron. This usually takes the form of long trains of spikes following the injection 
of short hyperpolarizing current pulses. Note that the essential action of Im inhibition is 
not necessarily the slow EPSP seen, but rather the facilitation of the neurons response to 
some other excitation. 2 M-current is also sensitive to the neuropeptide LHRH (Adams ef a/., 
1982b; Jan and Jan, 1983; see section 4.3). 

4.2.2 Neuronal Operation 

The main effect of M-current inhibition is a rise in excitability, while the siow, low-amplitude 
EPSP may be more of an epiphenomena. The operation implemented by this inhibition has 
thus the character of gain control, i.e. modulating the input-output properties of the cell on 
a long-lasting time basis. By inhibiting I M , the neuron has lowered its threshold to synaptic 
input. Figure 20b graphically demonstrates the possible effect of /M-current inhibition on 
the / -1 curve of the cell. Whiie the absolute current threshold remains unchanged, the 
slope of the linear relation between / and I increases. 


4.3 Action of ’’Neurotransmitters" over Long Distances 


4.3.1 Biophysical Mechanism 


There is good experimental evidence that certain ’’neuroactive" substances, in particular 
neuropeptides, are capable of influencing the electrical behavior of neurons ”at a distance", 
i.e. over distances large in comparison with the metric employed for measuring the 
dimensions between the conventional presynaptic-postsynaptic elements. 3 Examples are 
the action of a luteinizing-hormone-releasing-hormone (LHRH) like peptide in bullfrog 
autonomic ganglion (Jan and Jan, 1983), FMRFamide-iike and Proctolin-like peptides in the 
stomatogastric ganglion of crustaceans (Marder and Hooper, 1984; Hooper and Marder, 
1984). egg-laying hormone in Aplysia (Branton, Mayeri, Brownell and Simon, 1978) and the 

detailed computer simulations of the bullfrog sympathetic neuron, including its seven different 
currents, calcium and potassium buffers, diffusion and membrane pumps, demonstrate that the slow 
EPSP and the enhanced susceptibility of the neuron to current inputs is due to the inhibition of 
two currents, Im and a smaller, calcium-dependent, non-inactivating current mediating the long 
hyperpolarization following single spikes, l A ur (Koch and Adams, in preparation). 

3 Schmitt (1984) proposes the generic term informational substances to name these and similar 
chemical mediators. 
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F.gure 20. (a) The intracellular potential in a bullfrog sympathetic neuron following a single 

preganglionic stimulus. The fast EPSP, and the thereby induced action potential, is mediated by a 
n '^° K ' niC A k ^ h receptor while slow EPSP results from the action of ACh on a muscarinic receptor 
inhibiting M-current. Adapted from Adams and Brown, 1982. (b) One possible effect of M-current 
inhibition on the steady-state firing frequency of a neuron. Note that the current threshold remains 
unaffected, but that the total response of the cell to synaptic input increases. 


proposed nonsynaptic release of norepinephrine and serotonin from the diffusely branching 
axons originating in the locus coeruleus and the raphe nucleus (Dismukes, 1979; Moore and 
Bioom, 1978). In order to understand some of the issues involved, we will briefly examine 
the action of a LHRH-like peptide in the bullfrog. 


Stimulation of cholinergic preganglionic fibers in the frog sympathetic ganglion gives rise 
to three different postsynaptic responses: the fast, nicotinic-mediated EPSP and the slow, 
muscarinic-mediated EPSP (as discussed in the previous section). A third synaptic potential, 
termed the late slow potential, lasts for several minutes and is not mediated by ACh (Jan 
and Jan, 1982 and 1983). Its main action is to increase the excitability of the neuron, 
similar to M-current inhibition. In fact, the M channel probably is an important component 
generating the late slow EPSP, although it cannot fully account for ali of its ionic basis 
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(Adams et al., 1982a,b). It is believed that the late slow EPSP is mediated by a peptide 
structurally very similar to LHRH. This peptide is contained and released from the same 
preganglionic fiber as ACh. Stimulating a fiber presynaptic to a C neuron gives rise to both 
the ACh mediated synaptic event and to the late slow EPSP. B cells, though not in any 
synaptic contact with the fibers presynaptic to C cells, also show the late slow EPSP upon 
stimulation of the preganglionic C fibers. Thus, the LHRH-like peptide must diffuse for tens 
of microns from its release site before reaching the surface of B cells. The longevity of the 
late slow EPSP is not limited by the diffusion time but is a consequence of the long lifetime 
of the peptide. 

Three points are particularly noteworthy in this example and can be generalized to similar 
cases in other systems. The influence of the peptide on the target cell is of long duration, 
typically much longer than the millisecond range of action potentials, and contingent 
upon previous excitation in the target neuron. The last attribute is crucial to the possible 
modulatory function of peptides (see the discussion by L.L. iversen in Dismukes, 1979). 
Thus, M-current inhibition increases the excitability of the neuron without, or only with little, 
concomittant rise in voltage. Similarly, substance P acts on Renshaw cells in the spinal cord 
to antagonize the nicotinic actions of ACh, while having no effect when administered alone 
(Belcher and Ryall, 1977). Lastly, the sphere of influence nf these modulatory substances is 
not confined to the synaptic cleft, but may extend well above the 10 pm. range. Their speed 
of action is usually limited, in the absence of any active transport process, by diffusion, that 
is distance is proportional to the square (or cubic) root of the elapsed time. It is important 
then, that these substances are not degraded by enzymes during their diffusion from the 
release site to the receptor. , : 

Increasing evidence from immunohistochemical studies reveals that neurons may contain one 
or more neuropeptides in addition to a monomine neurotransmitter such as noradrenaline or 
ACh (Hoekfeltef a/., 1980; for an overview see Lundberg and Hoekfelt, 1983). Concomittantly, 
iontophoretic experiments demonstrate the presence of more than a single population of 
receptors on a particular postsynaptic neuron. Cells in the frog sympathetic ganglion 
respond to ACh, substance P and LHRH (Jan and Jan, 1982) while locus coeruleus neurons 
bear both opiate and a-adrenoreceptors (Starke, 1981). Since ail neuropeptides are made 
up of amino acids, the total number of possible peptides could be astronomically large, 
depending on the peptide chain length. Lastly, neuropeptides differ fundamentally from ACh, 
biogenic amines and amino acids since they are not recycled or reuptaken. Peptidergic 
neurons rely entirely on the synthesis of the peptide in the cell body and on its axonal 
transport to the synaptic terminal. This might well have functional consequences under 
conditions of tonic firing. 


4.3.2 Neuronal Operation 


49 



Koch and Poggio 


Biophysics of Computation 






/"“N 


We propose that the main action of neuropetides and similar substances is a long-term 
modulation of cellular excitability without influencing directly neuronal excitability (see figure 
20b). Applying the neuropeptide in the absence of any synaptic input will have little effect 
on the behavior of the neuron. The underlying biophysical mechanisms can be likened 
to a gain control. The major differences to the "conventional" mode of fast and focal 
chemical interaction is that neuropeptides can activate large areas in the CNS by diffusing 
throughout the extracellular matrix and that their time-course is typically two to three 
orders of magnitude larger than the time-course of postsynaptic conductance changes 
or of impulse initiation. This restricts the use of neuropeptides and similar substances to 
circumstances where the global properties of a given hardwired circuit need be modified 
to reflect a different situation requiring a different kind of information processing (see for 
instance Marder, 1984). Owing to their time-course, it seems unlikely that neuropeptides 
underly fast computations required for visual perception and motor control. 

4.3.3 An Example of a Computation : , 


Recent efforts to plan and build massive parallel computers has led to the growing realization, 
that one major challenge facing the computer architecture designer is the problem of 
routing information efficiently among the individual processors, i.e. with the least amount 
of time and connections (Hiilis, 1981; Poggio, 1984). The cortex, a computational structure 
comprising roughly 10 u individual processors operating in parallel, probably faces a similar 
problem. How is information routed among neurons without quickly exceeding space by 
connecting every neuron with every other neuron? One possible mechanism might involve 
neuropeptides. The basis of the mechanism is illustrated in figure 21. Let us assume that 
a particular sensory neuron is presynaptic to a large number of neurons, i.e. establishes 
conventional synapses with them. In the absence of any modulatory input, electrical activity 
in the sensory neuron evokes some activity in the corresponding postsynaptic elements. In 
the presence of a particular neuropeptide A. for which only a subset M A of all neurons have 
a receptor, the excitability of all neurons in M A is enhanced. The neuropeptide is released 
from a local, diffusely branching interneuron. Activity in the sensory neuron will now evoke 
a much stronger response in the affected neurons than before and the information about 
the occurance of a specific stimulus has been routed to a certain subset of neurons. If 
another population of neurons has receptors to a different peptide, the information is routed 
to a different target. In order for such an addressing scheme to work efficiently, a large 
number of different peptides is required in order to "program" different paths. The number 
and location of neurons which could be addressed in this way depends critically on the 
transport mechanism of the neuropeptide, i.e. on diffusion in the extracellular space. Thus, 
addressing works by selectively and temporarily increasing the gain of the active membrane 
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Figure 21. A highly speculative addressing scheme in a closely packed neuronal ganglion. The 
axon (heavy line) is presynaptic to a variety of cells which project outside the ganglion. Each one of 
these cells has receptors for one or two different neuromodulators, such as neuropeptides (A, B and 
C). They are released by local interneurons projecting diffusely throughout parts of the ganglion. 
Once a specific neuromodulator has been released, it diffuses to its receptor sites, enabling the 
postsynaptic cell to fire much more vigorously in response to a presynaptic input than before. 
Thus, information from the axon has been routed to a subset of ail its postsynaptic neurons. This 
mechanism requires a large number of neuropeptides or similar substances, which may diffuse for 
many microns before binding to a receptor. Moreover, neurons are assumed to have receptors for 
a few of these substances. 

properties of the class of neurons to be addressed, without actually exciting them. This 
solution to the addressing problem is similar to the traditional telephone system, where 
connections are made and broken as required for exchanging information. It is conceivable 
that neuropeptides or other modulators may route information within the branched axonal 
tree of a single neuron by controlling in a differentia! way the active membrane properties 
of the axonal branches. 
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5. Biophysics of Computation 


From the point of view of information processing, an important difference among the 
biophysical mechanisms we have reviewed above is likely to become increasingly clear. 
Action potential generation and propagation, synaptic transmission and the interaction 
between synaptic inputs usually occurs on a timescale of milliseconds, what we term the 
primary, logical timescale. The lower limit on this time-scale is given by the speed of 
propagation of the electrical potential in axons and other excitable tissues and by the 
chemical kinetics of the binding of receptors and transmitters. The primary timescale can 
be likened to the basic cycle time of a digital computer. However, some of the newly 
discovered currents, like for instance the slow, muscarinic-mediated, cholinergic excitation 
(Brown, 1983) or the effects of neuropeptides on the firing behavior of neurons, take many 
hundreds of millisecond to be noticeable, and may last many seconds and even minutes. 
Their use in information processing would be one of modulating the existing, hardwired 
circuitry, adapting the circuit to different stimuli or to different modes of operation (e.g. 
Marder, 1983). They act in what we call the secondary, modulatory timescale. Biophysical 
mechanisms operating in the primary time range most likely underly visual perception and 
the computation required for motor output. Mechanisms acting on the modulatory timescale, 
on the other hand, may, increase the excitability of a neuronal population following some 
stimuli alerting the animal to a potential danger, or may mediate sensory aftereffects or 
habituations. While the biophysical mechanisms in the primary timescale correspond to the 
physical mechanisms underlying the operation of transistors and gates, there are at present 
no counterparts to the biophysical mechanisms operating in the modulatory time range. 

We still know too little about the computational style used by nervous systems. As a 
consequence there is a large gap between computational 2theories of vision and motor 
control and their possible implementation in neural hardware. The model of computation 
provided by the digital computer is clearly unsatisfactory for the neurobiologist, given the 
increasing evidence that neurons are complex devices, very different from the simple digital 
switches suggested by the McCulloch and Pitts (1943) neurons, it is especially difficult to 
imagine how networks of neurons may solve the equations involved in vision algorithms in 
a way similar to digital computers. 

Recently, Poggio and Koch (1985) have suggested a powerful analog model of computation 
in electrical or chemical networks for a large class of vision problems, that maps more 
easily into biologically plausible mechanisms of the type we discuss in this paper. Their 
suggestion is based on a new theoretical framework for early vision - regularization theory - 
introduced by Poggio and Torre (1985). Regularization methods solve early vision problems 
in terms of variational principles of a specific type. Poggio and Koch begin by recognizing 
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that analog electrical networks are a natural hardware for computing the class of variational 
principles suggested by regularization analysis. Because of the well-known isomorphism 
between electrical and chemical networks (see for instance Busse and Hess, 1973 or Eigen, 
1974) that derives from the common underlying mathematical structure (i.e. Kirchoff’s laws), 
appropriate sets of chemical reactions can be devised, at least in principle, to “simulate" 
exactly the electrical circuits (figure 22). 

Electrical and chemical systems of this type therefore offer a computational model for 
early vision that is quite different from the digital computer. Equations are “solved" in an 
implicit way, exploiting the physical constraints provided by Kirchoff’s laws. It is not difficult 
to imagine how this model of computation could be extended to mixed electrochemical 
systems by the use of transducers, such as chemical synapses, that can decouple two 
parts of a system, similarly to operational amplifiers (Poggio and Koch, 1985). 

Could neural hardware exploit this model of computation? Several researchers had proposed 
in the past analog models of computation for the nervous system, most notably Jerry Lettvin. 
The evidence reviewed in this paper shows that for implementing electrical networks in 
equivalent neuronal hardware, one can draw upon a large number of elementary circuit 
elements. Patches of neuronal membrane or cytoplasm can be treated as resistances and 
capacitances. Voltage sources may be mimicked by synapses on dendritic spines or thin 
dendrites (Koch and Poggio, 1983b), whereas synapses on large dendrites may act as 
current sources. Chemical synapses could effectively serve to decouple different parts 
of a network (see Poggio and Koch, 1985). Chemical processes such as the reactions 
associated with postsynaptic effects or with neuropeptides could also be thought as part 
of a complex electrochemical network. Obviously, the analogy cannot be taken too literally. 
We are convinced, however, that the style of computation represented by analog circuits 
represents a very useful model for neural computations. 


In the brief history of computer technology, several different physical effects have been 
exploited to perform elementary operations, mostly digital. Within a single, very specific 
technology, however, typically a small number of physical mechanisms and corresponding 
operations are used, as for instance the inverter and NAND gating in MOS technology. 
Most of the more complex operations are synthesized starting from these two basic ones. 

It would be very surprising if evolution were to choose a very small number of basic 
mechanisms in the nervous system. This is certainly the “old" view, in which the only 
operation was a thresholding operation, implemented through the mechanism of spike 
initiation. The evidence accumulated over the last several years by research on membranes 
and synapses, has made abundantly clear, however, that several different mechanisms are 
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Figure 22. (a) A resistive network solving a particular visual task: computing the smoothest 

velocity field of a moving contour (Hildreth, 1984). The input data, injected current of amplitude /<, 
corresponds to the measured perpendicular velocity component v- 1 . The solution of the network, 
the voltages Vi, correspond to the unknown normal velocity components vj. Sampling the output, 
i.e. the voltage, between nodes corresponds to linear interpolation between the node values. Notice, 
that the value of the conductance <?,, varies from location to location, (b) A chemical network which 
is formally equivalent to the electrical circuit shown in (a). A substance A, the concentration of 
which corresponds to the desired i> T , diffuses along a cable while reacting with an extracellular 
substance S (first order kinetics). The corresponding On-rate k f varies from location to location. 
This could be achieved, for instance, by a differential concentration of an enzyme. The inputs 
are given by the influxes of substance A. (c) This scheme illustrates an hypothetical neuronal 
implementation of the electrical and chemical circuit shown above. A dendrite, acting as both pre- 
and postsynaptic element, has a membrane resistance that can be locally controlled by suitable 
synaptic input, preferentially from channels with f reversal potential close to the resting potential 
of the dendrite. The output, the voltage V,, is sampled by dendro-dendritic synapses. Adapted from 
Poggio and Koch, 1985. 
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likely to be exploited by the brain for information processing (Schmitt, 1979; Bullock, 1979). 
It seems unlikely, on the other hand, that the number of basic mechanisms is much larger 
than, say a dozen or so. This general class of mechanisms includes 

• Voltage-dependent channels 

• Neurotransmitter-dependent channels 

• Modifications of subcellular components 

• Spatio-temporal integration in neuronal structures 

Our approach — to identify and characterize some of the basic mechanisms for 
information processing — may provide powerful constraints for interpreting morphological 
and physiological data, and for connecting them with computational theories. Clearly, this 
plan can only be successful if the number of basic biophysical mechanisms in information 
processing is small. 

The final answer to these questions can only be obtained with experimental methods. 
A case in point is provided by our conjecture about shunting inhibition being a basic 
mechanism for implementing operations of the analog AND-NOT type. We have isolated a 
few computations, such as the computation of directional motion, where shunting inhibition 
may have a critical role. We have shown with theoretical methods and computer simulations 
based on anatomical data, that the properties of the biophysical mechanisms are consistent 
with our conjecture in the case of retinal ganglion cells. We have also made specific 
experimental predictions. Several of them are supported by recent data. If our conjecture 
turns out to be correct in the case of directional selectivity of retinal ganglion cells, the 
next task will be to demonstrate that the same mechanism is, or is not, used in other parts 
of the nervous system for other computations. 
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